mirror of
https://github.com/supabase/supabase.git
synced 2026-10-11 12:25:05 +03:00
fix-data-api-error
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Sean Oliver
|
b041ebfaa5 | feat(www): Phase 2 — enable cookie stamping on /dashboard and /docs paths (#43677) | ||
|
Sean Oliver
|
8ebbad3a5b |
feat(growth): expand www middleware to /dashboard and /docs (Phase 1 - instrumentation only) (#43413)
## Problem The `_sb_first_referrer` cookie isn't working. The www middleware matcher explicitly excludes `/dashboard` and `/docs`, so the cookie never gets stamped for Studio or Docs traffic. PostHog confirmed: only 1 event with `first_referrer_cookie_present=true` out of ~46.5M Studio pageviews in the last 7 days. ## Background: what the matcher does In Next.js, the `matcher` config controls which incoming requests the middleware function even runs on. If a path doesn't match, the middleware is skipped entirely — the request passes through untouched. If it matches, the middleware runs and can mutate the response (set cookies, headers, etc.). This matters because Studio's SPA navigation works via silent `/_next/data/` JSON fetches. If middleware runs on those requests and returns `NextResponse.next()` with any mutations, it breaks those fetches and causes full page reloads instead of client-side transitions. ## What we tried before | PR | www runs on `/dashboard`? | Studio `proxy.ts` runs on all routes? | Result | |---|---|---|---| | **#42768** (Attempt 1) | ✅ Yes — and also intercepts `_next/data` | ✅ Yes — `matcher` config removed, stamps cookie everywhere | Full page reloads in Studio | | **#43129** (Full revert) | ❌ No — www middleware deleted entirely | ❌ No — restored to `matcher: '/api/*'` only | Back to baseline, no cookie stamping anywhere | | **#43153** (Attempt 2) | ❌ No — `/dashboard` explicitly excluded | ✅ Yes — `matcher` config removed again, stamps cookie everywhere | Full page reloads in Studio again | | **#43189** (Attempt 3) | ❌ No — same as #43153 | ✅ Yes — `matcher` config still removed, cookie stamping made conditional | Still broken | | **#43190** (Ivan's fix) | ❌ No — `/dashboard` still excluded | ❌ No — restored to `matcher: '/api/*'` only | Works — but cookie never stamps for `/dashboard` traffic | | **#43413** (this PR) | ✅ Yes — sets diagnostic cookie only, no attribution stamping yet | ❌ No — unchanged, still `matcher: '/api/*'` only | ❓ Untested in prod | The common factor in every failure: Studio's `proxy.ts` ran on all routes (including `_next/data` requests), which broke SPA navigation. This PR is the first one that runs www middleware on `/dashboard` while Studio's `proxy.ts` stays in its original narrow `/api/*` scope. ## What changed This is Phase 1 of a two-phase rollout. We remove `dashboard|docs` from the matcher's negative lookahead so www middleware runs on those paths — but instead of stamping cookies, we set a short-lived (60s) diagnostic cookie `_sb_mw_diag` on `/dashboard` and `/docs` requests. The diagnostic cookie encodes `hit=1&would_stamp={0|1}&has_cookie={0|1}`, which Studio telemetry reads on the initial pageview and reports to PostHog as `mw_diag_hit`, `mw_diag_would_stamp`, and `mw_diag_has_existing_cookie` properties. A cookie rather than a header because response headers aren't readable by JS. It also tests the actual Set-Cookie mutation path that Phase 2 will use (which is what Next.js issue #41885 is specifically about). Phase 1 answers two key questions before we commit to Phase 2: 1. Does expanding the matcher break Studio SPA navigation? 2. What % of /dashboard arrivals would get a first-referrer cookie stamped in Phase 2? ## Phase 2 readiness criteria **Important caveat**: `mw_diag_*` data reflects consented users only and may under-represent first-visit anonymous traffic. The Phase 2 decision should account for this — the actual middleware execution rate is likely higher than what PostHog reports. ### PostHog query spec **Middleware execution rate**: Of all Studio initial pageviews on `/dashboard` or `/docs` paths, what percentage have `mw_diag_hit = true`? Expected: >= 90%. Below 70% warrants investigation (could indicate edge caching bypassing middleware, or a matcher configuration issue). ``` Filter: event = "$pageview" AND (current_url contains "/dashboard" OR current_url contains "/docs") Breakdown: mw_diag_hit (true vs null/missing) Metric: count(mw_diag_hit = true) / count(all) * 100 ``` **Would-stamp rate**: Of events with `mw_diag_hit = true`, what percentage have `mw_diag_would_stamp = true`? This tells us what percentage of Phase 2 traffic would actually get a cookie stamped. No hard threshold — unexpected values (< 5% or > 95%) suggest a logic bug worth investigating before Phase 2. ``` Filter: event = "$pageview" AND mw_diag_hit = true Breakdown: mw_diag_would_stamp (true vs false) Metric: count(mw_diag_would_stamp = true) / count(all) * 100 ``` **Existing cookie rate**: Of events with `mw_diag_hit = true`, what percentage have `mw_diag_has_existing_cookie = true`? This tells us how many users already have the cookie from a prior www visit. ``` Filter: event = "$pageview" AND mw_diag_hit = true Breakdown: mw_diag_has_existing_cookie (true vs false) Metric: count(mw_diag_has_existing_cookie = true) / count(all) * 100 ``` ### Go / no-go threshold table | Signal | Go | Investigate | No-Go | |---|---|---|---| | `mw_diag_hit` rate (% of /dashboard+/docs pageviews) | >= 90% | 70-90% | < 70% | | SPA navigation errors (Sentry / Vercel logs) | No increase | < 0.1% increase | > 0.5% increase | | Middleware p99 latency (Vercel function logs) | < 50ms added | 50-100ms | > 100ms | | Sample volume in first 24h | > 1,000 events | 100-1,000 (extend window) | < 100 (insufficient data) | ## Changes - `apps/www/middleware.ts`: Removed `dashboard|docs` from matcher; added `isDashboardOrDocs` guard that sets `_sb_mw_diag` diagnostic cookie instead of stamping attribution - `apps/www/middleware.test.ts`: 12 tests covering cookie stamping on www paths, diagnostic cookie encoding for all scenarios (external referrer, direct nav, internal referrer, existing cookie) - `packages/common/first-referrer-cookie.ts`: Exported `MW_DIAG_COOKIE_NAME`, `MwDiagData` type, and `parseMwDiagCookie()` helper - `packages/common/telemetry.tsx`: Reads `_sb_mw_diag` on initial Studio pageview; reports `mw_diag_*` properties to PostHog ## Testing Unit tests pass (13/13 www, 31/31 first-referrer-cookie). Production validation needed: - [ ] Studio SPA navigation works (tab changes, SQL editor, no full page reloads) - [ ] PostHog shows `mw_diag_hit = true` on Studio initial pageviews - [ ] `mw_diag_would_stamp` distribution looks reasonable before enabling Phase 2 - [ ] Monitor 24h before Phase 2 Ref: GROWTH-625 / GROWTH-668 |
||
|
Sean Oliver
|
75ec7c6e6b |
feat(growth): re-land first-referrer cookie attribution with fixed middleware matchers (#43153)
## Summary Re-lands the first-referrer cookie feature from #42768 (reverted in #43129) with middleware matcher fixes that prevent Studio traffic interference. **Tracks:** [GROWTH-651](https://linear.app/supabase/issue/GROWTH-651) ## What changed New shared module in `packages/common/first-referrer-cookie.ts` that handles stamping and parsing a first-referrer cookie (referrer, UTMs, click IDs, landing URL). Each app's middleware calls `stampFirstReferrerCookie` on the edge response — www and docs are the primary entry points, Studio is a fallback for direct visits with UTMs. On the telemetry side, `handlePageTelemetry` now takes an options object instead of positional args, reads the cookie on initial pageview, and overrides the referrer if the cookie captured an external source but the current referrer is internal (i.e., the user navigated cross-app). Also sends `first_referrer_cookie_present`/`consumed` properties so we can observe the handoff in PostHog. The docs middleware matcher was broadened from `/reference/:path*` to all docs pages so we stamp cookies site-wide, not just on reference paths. ## Root cause of original revert Two layers: 1. **Matcher gap**: www middleware ran on `/dashboard/*` traffic in prod due to Vercel Multi-Zone architecture (www is the gateway for `supabase.com`, proxying `/dashboard` → Studio, `/docs` → Docs). Middleware runs *before* rewrites, so www middleware executed on all proxied traffic. 2. **`_next/data` interception**: The matcher didn't exclude `_next/data` paths. Client-side navigation in Next.js fetches JSON via `/_next/data/...` — middleware intercepted these, returned `NextResponse.next()` with cookie mutations (which processes through the middleware response pipeline), and this interfered with the JSON responses, causing full page reloads in the SQL editor. ## How this PR fixes it | Fix | Detail | |---|---| | Exclude `_next/data` | All three matchers (`www`, `docs`, `studio`) exclude `_next/data` via negative lookahead | | Exclude `dashboard` + `docs` from www | www middleware no longer runs on proxied app traffic | | `/api/` path guard in Studio | Broadened matcher requires explicit path check for API route filtering | | `NextResponse.next()` semantics | Cookie stamping only happens on matched paths; unmatched paths never enter middleware | ### `NextResponse.next()` vs `undefined` nuance Returning an explicit `NextResponse.next()` with cookie mutations processes through Next.js's middleware response pipeline (headers are merged, cookies are set). Returning `undefined` (i.e. the request never matches the matcher) lets Next.js handle the request completely untouched. The matcher exclusions ensure `_next/data` and proxied app paths never enter middleware at all. ## Testing - ✅ 22 unit tests for shared cookie utilities (all pass) - ✅ Studio prod build succeeds, middleware recognized as `ƒ Proxy (Middleware)` - ✅ Playwright validation: client-side navigation works across 3 page transitions, `_next/data` requests return 200 OK without middleware interception, no full-page reloads - ❌ www/docs SSG builds require platform backend services (expected — same as master) |
||
|
Ivan Vasilov
|
85e6b1143f |
chore: Revert "fix: persist first referrer across app boundaries (#42768)" (#43129)
This reverts commit https://github.com/supabase/supabase/commit/04e63dfb2e00c43f8c57e538b8056aa647f4e5b4 since it was causing the Studio app to rerender the full page on every link navigation. |
||
|
|
04e63dfb2e |
fix: persist first referrer across app boundaries (#42768)
## Summary Fixes GROWTH-625. Preserves first-touch attribution across app boundaries by persisting external referrer context at the edge and consuming it on Studio's initial pageview. When users come from an external source to www/docs and then navigate to Studio, Studio often only sees the internal `supabase.com` hop. This change preserves the original external context so first-touch attribution is retained. ## What changed - **Shared first-referrer cookie utilities** (`packages/common/first-referrer-cookie.ts`): - `isExternalReferrer`, `buildFirstReferrerData`, `serializeFirstReferrerCookie`, `parseFirstReferrerCookie` - `hasPaidSignals` — detects click IDs (gclid, fbclid, etc.) and paid utm_medium values - `shouldRefreshCookie` — centralizes stamp-or-skip decision for all apps - `stampFirstReferrerCookie` — shared middleware helper used by all apps (extracted from duplicated inline logic) - **Edge middleware on all apps** — stamps cookie for external visitors, refreshes on paid signals: - `apps/www/middleware.ts` (simplified to use shared helper) - `apps/docs/middleware.ts` (simplified to use shared helper) - `apps/studio/proxy.ts` (integrated into existing proxy file) - **Docs middleware matcher** — broadened from `/reference/:path*` to all non-static paths so the first-referrer cookie is stamped on all docs pages, not just reference paths - **Telemetry** — Studio consumes cookie on initial pageview (`packages/common/telemetry.tsx`). `handlePageTelemetry` refactored from 7 positional params to an options object for readability. - **Tests** — 22 unit tests covering all utilities and edge cases (including direct-navigation scenario) ## Behavior - Writes `_sb_first_referrer` cookie when: - cookie is not already set and request has an external referrer, OR - cookie exists but incoming URL has paid traffic signals (click IDs or paid utm_medium) - Cookie: 365-day TTL, `domain=supabase.com`, `sameSite=lax`, `secure=true` in production - On first Studio pageview, if current referrer is internal and cookie has external context: - use persisted external referrer - apply persisted UTM/click-id/landing-url attribution props - Measurement properties: `first_referrer_cookie_present`, `first_referrer_cookie_consumed` ## Manual testing 1. Visit `supabase.com/pricing?utm_source=google&utm_medium=cpc` from an external referrer (or use DevTools to set a `Referer` header) 2. Check `_sb_first_referrer` cookie is set in Application > Cookies 3. Navigate to Studio (`supabase.com/dashboard`) 4. In PostHog (or browser network tab), verify the first `$pageview` event has: - `first_referrer_cookie_present: true` - `first_referrer_cookie_consumed: true` - `$utm_source: "google"`, `$utm_medium: "cpc"` - `$referrer` points to the external source, not `supabase.com` 5. Verify subsequent route changes do NOT include `first_referrer_cookie_*` properties ## Review feedback addressed - Added `secure: true` flag on production cookies (Pam's first comment) - Fixed inaccurate JSDoc on `utms` field — keys retain `utm_` prefix (Pam's fourth comment) - Added test coverage for edge cases: malformed URLs, multi-cookie headers, http:// referrers (Pam's sixth comment) - Docs matcher broadening: fast-path exit on cookie-exists check keeps overhead minimal, exclusion list is correct - Extracted shared middleware helper to eliminate duplication across 3 apps - Refactored `handlePageTelemetry` from positional params to options object - Removed redundant null check in `hasPaidSignals` - Added direct-navigation test case - Deleted dead `apps/learn/middleware.ts` - Fixed studio build: integrated cookie stamping into existing `proxy.ts` (Next.js 16 rejects both middleware.ts and proxy.ts) --------- Co-authored-by: pamelachia <26612111+pamelachia@users.noreply.github.com> Co-authored-by: Pamela Chia <pamelachiamayyee@gmail.com> |