## Problem
We often get errors while starting the local supabase instance on CI. We
added diagnostics but they are targeting the wrong port.
## Solution
Fix the diagnostics logs
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Granted additional CI credentials and added cloud role assumption plus
container registry login before running self‑hosted end-to-end tests.
* Improved workflow diagnostics to check an alternate local database
port (54322) and show clearer messages when the service is not
listening.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
We run the e2e tests across two shards to reduce their duration.
However, one takes longer than the other.
## Solution
Use a currently hidden Playwright feature to balance their load
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded Playwright test dependency to a newer version for refreshed
test tooling and fixes.
* Adjusted end-to-end test shard weighting to rebalance distribution
across parallel runs.
* **Tests**
* Stabilized several end-to-end editor interactions to make E2E flows
more reliable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
The Supabase CLI sometimes fails to start because the postgres port is
already used:
> failed to start docker container: Error response from daemon: failed
to set up container networking: driver failed programming external
connectivity on endpoint supabase_db_supabase
(10587f26e7287c8086fd27db12be7aac4afe1f3fc521c9d7a8dfe109993c995d):
failed to bind host port for 0.0.0.0:54322:172.18.0.2:5432/tcp: address
already in use
## Solution
None yet, debugging
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced end-to-end test workflow with added pre-start and failure
diagnostics for better observability and troubleshooting of test runs.
* Updated the e2e setup CLI script to skip certain local services by
default and clarify script formatting for maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
Since the upgrade to NextJS 16.1.6, many e2e tests run fails with
timeout, probably when building or starting the studio
## Solution
- [x] Add dedicated steps for supabase setup and NextJS build to ease
debugging
- [x] Fix flakyness
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.
YES
## What kind of change does this PR introduce?
Our runners is running out of memory and CPU when running the build.
Just trying to bump up the blacksmith runner so they match the build
machine in Vercel.
## Problem
e2e tests are still flaky and not as fast as they could be
## Solution
- [x] Reset supabase instance: this makes the database visualiser tests
faster as there are less tables to screenshot
- [x] Improve the single file setup utilities so that they never block
local tests by cleaning them up before starting
- [x] Disable animations while running the tests (less time waiting for
animations to complete
- [x] Add utility functions that help reproducing flaky tests locally
* updated commands and expose ai key locally
* added tests for AI assistant
* added OPEN_API_KEY for e2e test suite
* updated log drain options
* updated README
* a new hope
* run tests in ci against cli mode
* summary
* try vercel action to run e2e against studio self hosted preview
* believe
* debug
* gh pages artifact
* test
* rm pages step
* fix automation bypass missing
* continue on error
* only install necessary deps for CI
* fix bypass
* remove
* fail job if test fails
* disable customer query if is_platform false
* vercel check
* fix var name, make comment update instead
* check bypass on runtime
* add env var
* fix tests going to project ref instead of default
* fix
* better dates in comment
* Update E2E test workflow to include flaky test detection and improve summary output
* fix
* fix dumb mistake
Gildas Garcia
