From e846d45ce64207b952a4df44ac8b480ea0abb27e Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Tue, 11 Aug 2026 13:57:01 +0000 Subject: [PATCH] chore(studio): retry flaky unit tests in CI (#48939) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _Requested via [Slack thread](https://supabase.slack.com/archives/C063LNYJJKS/p1786454906416269?thread_ts=1786454906.416269&cid=C063LNYJJKS)_ ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Chore / CI reliability. One-line config change to `apps/studio/vitest.config.ts`. ## What is the current behavior? **Before:** the `Studio Unit Tests & Build Check` workflow sometimes goes red on `master` for no reason anyone can act on. Since 2026-07-29 it failed 3 out of 82 test executions (3.7%), every time at job `test (1)`, step `Run Tests`. Every one of those three passed on a re-run with no code change: - https://github.com/supabase/supabase/actions/runs/31495760766 (`4587d177`, Aug 11) - https://github.com/supabase/supabase/actions/runs/31409667566 (`b04d1485`, Aug 10) - https://github.com/supabase/supabase/actions/runs/31203465483 (`777c02c2`, Aug 7) Each failure also posts a Slack alert to #team-frontend-alerts via `.github/workflows/studio-master-alert.yml`, so someone gets pinged, opens the run, clicks re-run, and it goes green. ## What is the new behavior? **After:** a test that fails in CI gets up to two more attempts before the job is marked failed. A genuinely broken test still fails all three attempts and still goes red. Locally nothing changes — the first failure is the result you see, so you are never waiting on retries while debugging. ## Additional context **How:** added `retry: IS_CI ? 2 : 0` to the `test` block of `apps/studio/vitest.config.ts`, with `const IS_CI = !!process.env.CI` matching the pattern already used in `e2e/studio/playwright.config.ts:51` (`retries: IS_CI ? 5 : 0`). **Known limitation — we do not know which test is flaking.** The GitHub Actions log downloads for those three runs were not retrievable, and the API only surfaces `Process completed with exit code 1`. So this treats the symptom without naming the cause. The natural follow-up is to upload a JUnit or JSON vitest report as an artifact with `if: always()`, which would name the flaking test on the next failure. That is deliberately **not** in this PR — it is a workflow change and was scoped out. One more honest caveat: per-test retry only helps if the failure is an assertion or timeout inside a test. If the real cause is a worker crash or OOM, retrying will not save the run. That is a live possibility here — the workflow sets `NODE_OPTIONS: '--max_old_space_size=3072'` with the in-repo comment "Default is 2 GB, increase to have less frequent OOM errors", which says someone has already hit memory pressure in this job. So: worth landing as a cheap reduction in false alarms, but if the 3.7% does not drop, the report artifact is the next step rather than more retries. --- _Generated by [Claude Code](https://claude.ai/code/session_01U4338RsMYAc1uGuwTFNGBD)_ Co-authored-by: Claude Co-authored-by: Ali Waseem --- apps/studio/vitest.config.ts | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/apps/studio/vitest.config.ts b/apps/studio/vitest.config.ts index 4c64e6653ea..b0d1a68df0f 100644 --- a/apps/studio/vitest.config.ts +++ b/apps/studio/vitest.config.ts @@ -9,6 +9,8 @@ import { configDefaults, defineConfig } from 'vitest/config' // `setupFiles` live next to the test file itself. This forces them to always resolve correctly. const dirname = fileURLToPath(new URL('.', import.meta.url)) +const IS_CI = !!process.env.CI + export default defineConfig({ plugins: [ react(), @@ -24,6 +26,8 @@ export default defineConfig({ test: { globals: true, environment: 'jsdom', // TODO(kamil): This should be set per test via header in .tsx files only + // Retry flaky tests in CI only; failures locally should surface immediately. + retry: IS_CI ? 2 : 0, setupFiles: [ resolve(dirname, './tests/setup/polyfills.ts'), resolve(dirname, './tests/vitestSetup.ts'),