From 4c77ab5fef3ceae480697cb5b504a9c64d1df253 Mon Sep 17 00:00:00 2001 From: Sean Oliver <882952+seanoliver@users.noreply.github.com> Date: Thu, 14 May 2026 12:52:43 -0700 Subject: [PATCH] feat(telemetry): mirror org_count to PostHog person property (#45946) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature + two follow-on fixes — small, scoped to telemetry / experiment plumbing. ## What is the current behavior? PostHog feature flags evaluated in Studio only have access to the `gotrue_id` person property (set in `useTelemetryIdentify`) and the `organization`/`project` group associations from pageviews. Flags can't target users by org membership without a behavioral cohort, which refreshes on a ~hourly schedule and lags behind real-time signup state. This is blocking the rollout of the `dataApiRevokeOnCreateDefault` experiment ahead of the May 30 default-privileges breaking change — we need to target brand-new dashboard signups with no prior org membership, and there's no person property to filter on. ## What is the new behavior? Three changes, scoped tightly to make experiment targeting reliable for brand-new signups: ### 1. Mirror `org_count` to a PostHog person property (`apps/studio/lib/telemetry.tsx`) The Studio `Telemetry` component now mirrors the user's current org-list length to a PostHog person property `org_count` via `posthog.identify(user.id, { org_count })`. The effect: - Subscribes to `useOrganizationsQuery` (shares the same React Query cache as `useSelectedOrganizationQuery`, so no extra network requests). - Dedupes via a ref keyed on `{ userId, orgCount }` so we only call identify when the value actually changes — handles user-switch (logout/login as different user with same count) correctly. - Generic enough to be useful beyond this experiment — analytics segmentation by org membership, future flags that depend on multi-org behavior, etc. ### 2. Merge pre-init identify properties (`packages/common/posthog-client.ts`) The previous `pendingIdentification` slot was a single-write buffer — calling `posthogClient.identify()` before the PostHog SDK initialized would overwrite any prior queued identify. Latent until this PR added a second identify caller (`org_count`), which exposed the last-write-wins behavior on first-visitor-before-consent flows. Now merges properties across pre-init calls for the same user so both `{ gotrue_id }` and `{ org_count }` land on the person record when the SDK flushes. Caught during Codex review. ### 3. Gate the exposure event on `org_count` being present (`apps/studio/hooks/misc/useDataApiRevokeOnCreateDefault.ts`) `useTrackDefaultPrivilegesExposure` previously fired on the first non-undefined value of the `dataApiRevokeOnCreateDefault` flag. For brand-new signups, this races the `org_count` identify: the initial `/flags/` response (before targeting can match) returns the untargeted variant, the exposure locks it in via `hasTracked`, then our identify fires and a subsequent `/flags/` refresh updates the flag — but the exposure has already recorded the wrong variant. Fix: gate the exposure on `org_count` being present on the SDK person, subscribing via `onFeatureFlags` so we pick up the post-identify `/flags/` response. Adds `posthogClient.getPersonProperty` as the local-state reader. Without this, the experiment would have a ~5-15% noise floor on cohort assignment for new signups. ## Verification End-to-end verified locally against the staging PostHog project (34343): - Local Studio's PostHog SDK has `$stored_person_properties: { gotrue_id: , org_count: 1 }` after sign-in. - Both `$set` events landed server-side within ~300ms of each other, and the staging person record now shows `org_count = 1.0` with `gotrue_id` preserved. - Targeting query `person.properties.org_count == 1` works end-to-end against staging. ## Additional context Ref: [GROWTH-853](https://linear.app/supabase/issue/GROWTH-853) Targeting plan for the flag once shipped: `person.org_count == 1` plus a behavioral filter on recent `sign_up` event, at 5% rollout. ## Summary by CodeRabbit * **Chores** * Telemetry now records and syncs the user's organization count as an analytics person property and avoids redundant identifications when unchanged. * Analytics client now merges queued identification properties made before initialization and exposes a method to read stored person properties. * **Bug Fixes** * Tracking now waits for organization-count readiness before firing certain exposure events to prevent missing data. * **Tests** * Added/updated tests to cover person-property behavior and gating logic. [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/45946) --- .../useDataApiRevokeOnCreateDefault.test.ts | 24 +++++++++++++ .../misc/useDataApiRevokeOnCreateDefault.ts | 36 +++++++++++++++---- apps/studio/lib/telemetry.tsx | 20 +++++++++-- packages/common/posthog-client.ts | 32 +++++++++++++++-- 4 files changed, 101 insertions(+), 11 deletions(-) diff --git a/apps/studio/hooks/misc/__tests__/useDataApiRevokeOnCreateDefault.test.ts b/apps/studio/hooks/misc/__tests__/useDataApiRevokeOnCreateDefault.test.ts index d6e636fed41..c4e2ee97841 100644 --- a/apps/studio/hooks/misc/__tests__/useDataApiRevokeOnCreateDefault.test.ts +++ b/apps/studio/hooks/misc/__tests__/useDataApiRevokeOnCreateDefault.test.ts @@ -1,4 +1,5 @@ import { renderHook } from '@testing-library/react' +import { posthogClient } from 'common' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { @@ -25,6 +26,17 @@ vi.mock('@/lib/telemetry/track', () => ({ useTrack: vi.fn(), })) +vi.mock('common', async () => { + const actual = await vi.importActual('common') + return { + ...actual, + posthogClient: { + getPersonProperty: vi.fn(), + onFeatureFlags: vi.fn(() => () => {}), + }, + } +}) + describe('useDataApiRevokeOnCreateDefaultEnabled', () => { afterEach(() => { vi.restoreAllMocks() @@ -62,6 +74,11 @@ describe('useTrackDefaultPrivilegesExposure', () => { beforeEach(() => { vi.mocked(useTrack).mockReturnValue(track) + // Default: org_count is set on the person — most tests want to exercise + // the post-targeting-resolution behavior. The gate-blocked case has its + // own test below. + vi.mocked(posthogClient.getPersonProperty).mockReturnValue(1) + vi.mocked(posthogClient.onFeatureFlags).mockReturnValue(() => {}) }) afterEach(() => { @@ -74,6 +91,13 @@ describe('useTrackDefaultPrivilegesExposure', () => { expect(track).not.toHaveBeenCalled() }) + it('does not fire while org_count is missing from the SDK person', () => { + vi.mocked(usePHFlag).mockReturnValue(true) + vi.mocked(posthogClient.getPersonProperty).mockReturnValue(undefined) + renderHook(() => useTrackDefaultPrivilegesExposure({ surface: 'main', dataApiEnabled: true })) + expect(track).not.toHaveBeenCalled() + }) + it('fires once when the flag resolves to true on the main surface', () => { vi.mocked(usePHFlag).mockReturnValue(true) renderHook(() => useTrackDefaultPrivilegesExposure({ surface: 'main', dataApiEnabled: true })) diff --git a/apps/studio/hooks/misc/useDataApiRevokeOnCreateDefault.ts b/apps/studio/hooks/misc/useDataApiRevokeOnCreateDefault.ts index 077d63e2308..5299d6cdac9 100644 --- a/apps/studio/hooks/misc/useDataApiRevokeOnCreateDefault.ts +++ b/apps/studio/hooks/misc/useDataApiRevokeOnCreateDefault.ts @@ -1,4 +1,5 @@ -import { useEffect, useRef } from 'react' +import { posthogClient } from 'common' +import { useEffect, useRef, useState } from 'react' import { usePHFlag } from '../ui/useFlag' import { IS_TEST_ENV } from '@/lib/constants' @@ -28,28 +29,49 @@ type DefaultPrivilegesExposureOptions = | { surface: 'vercel' } /** - * Fires `project_creation_default_privileges_exposed` once per mount after the - * `dataApiRevokeOnCreateDefault` flag resolves. Gating on flag resolution keeps - * cohort attribution clean — users whose flag never resolves are not counted in - * either cohort. Deduplicated via ref so re-renders and mid-session flag flips - * don't re-fire. + * Fires `project_creation_default_privileges_exposed` once per mount after both + * the `dataApiRevokeOnCreateDefault` flag resolves AND the `org_count` person + * property has been set on the PostHog SDK. Waiting for both signals avoids + * locking in the variant on the initial /flags/ response, which races the + * org_count identify for brand-new signups — the exposure must reflect the + * targeting-aware flag value. Deduplicated via ref so re-renders and mid-session + * flag flips don't re-fire. */ export const useTrackDefaultPrivilegesExposure = (options: DefaultPrivilegesExposureOptions) => { const track = useTrack() const flag = usePHFlag('dataApiRevokeOnCreateDefault') const hasTracked = useRef(false) + const [orgCountReady, setOrgCountReady] = useState( + () => posthogClient.getPersonProperty('org_count') !== undefined + ) const { surface } = options const dataApiEnabled = options.surface === 'main' ? options.dataApiEnabled : null + // Mark ready once org_count appears on the SDK person. A /flags/ response + // received after our identify will have included org_count in the evaluation, + // so subscribing via onFeatureFlags is the right signal that the flag store + // reflects the targeting-aware value. + useEffect(() => { + if (orgCountReady) return + const check = () => { + if (posthogClient.getPersonProperty('org_count') !== undefined) { + setOrgCountReady(true) + } + } + check() + return posthogClient.onFeatureFlags(check) + }, [orgCountReady]) + useEffect(() => { if (hasTracked.current) return if (flag === undefined) return + if (!orgCountReady) return hasTracked.current = true track('project_creation_default_privileges_exposed', { surface, ...(dataApiEnabled !== null && { dataApiEnabled }), dataApiRevokeOnCreateDefaultEnabled: flag, }) - }, [flag, track, surface, dataApiEnabled]) + }, [flag, orgCountReady, track, surface, dataApiEnabled]) } diff --git a/apps/studio/lib/telemetry.tsx b/apps/studio/lib/telemetry.tsx index 3b90b4049b7..dddd789fe24 100644 --- a/apps/studio/lib/telemetry.tsx +++ b/apps/studio/lib/telemetry.tsx @@ -1,8 +1,9 @@ import * as Sentry from '@sentry/nextjs' -import { LOCAL_STORAGE_KEYS, PageTelemetry, useUser } from 'common' -import { useEffect } from 'react' +import { LOCAL_STORAGE_KEYS, PageTelemetry, posthogClient, useUser } from 'common' +import { useEffect, useRef } from 'react' import { useConsentToast } from 'ui-patterns/consent' +import { useOrganizationsQuery } from '@/data/organizations/organizations-query' import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization' import { API_URL, IS_PLATFORM } from '@/lib/constants' @@ -28,6 +29,21 @@ export function Telemetry() { const user = useUser() + // Mirror the user's org-list length into a PostHog person property so feature + // flags and analytics can segment by current org membership (e.g. "users in + // exactly one org") without behavioral-cohort lag. Only fires when the value + // changes. + const { data: organizations } = useOrganizationsQuery() + const lastSentRef = useRef<{ userId: string; orgCount: number } | null>(null) + useEffect(() => { + if (!user?.id || !organizations) return + const orgCount = organizations.length + const last = lastSentRef.current + if (last?.userId === user.id && last.orgCount === orgCount) return + lastSentRef.current = { userId: user.id, orgCount } + posthogClient.identify(user.id, { org_count: orgCount }) + }, [user?.id, organizations]) + useEffect(() => { // don't set the sentry user id if the user hasn't logged in (so that Sentry errors show null user id instead of anonymous id) if (!user?.id) { diff --git a/packages/common/posthog-client.ts b/packages/common/posthog-client.ts index acff08b0b9e..20d03887f49 100644 --- a/packages/common/posthog-client.ts +++ b/packages/common/posthog-client.ts @@ -170,8 +170,15 @@ class PostHogClient { if (!hasConsent) return if (!this.initialized) { - // Queue the identification for when PostHog initializes - this.pendingIdentification = { userId, properties } + // Queue the identification for when PostHog initializes. Merge properties + // across pre-init calls for the same user so callers don't clobber each + // other (e.g. useTelemetryIdentify sets gotrue_id, then a separate effect + // sets org_count — both should land when the SDK flushes). + const pending = this.pendingIdentification + this.pendingIdentification = + pending && pending.userId === userId + ? { userId, properties: { ...pending.properties, ...properties } } + : { userId, properties } return } @@ -252,6 +259,27 @@ class PostHogClient { return undefined } + /** + * Returns the current value of a person property as stored locally by posthog-js. + * Returns undefined if PostHog hasn't initialized or the property hasn't been set. + * Use this to gate behavior on whether a property has actually landed in the SDK + * (e.g., waiting for an identify to complete before evaluating flag-dependent UI). + * + * Person properties set via `identify(id, props)` are stored under the + * `$stored_person_properties` bucket in persistence — `get_property(key)` + * reads top-level super properties, not person properties, so we index in. + */ + getPersonProperty(key: string): unknown { + if (!this.initialized) return undefined + try { + const stored = posthog.get_property('$stored_person_properties') + if (!stored || typeof stored !== 'object') return undefined + return (stored as Record)[key] + } catch { + return undefined + } + } + /** * Returns a PostHog feature flag value directly from the client-side SDK. * Use this for www/docs pages where server-side evaluation lacks full person context.