feat(www,docs): serve marketing pages as .md, advertise via link rel=alternate (#45277)

## Summary Adds `/<page>.md` routes for 10 marketing/product pages (homepage, auth, database, edge-functions, realtime, storage, vector, pricing, modules/cron, modules/queues) so AI agents can fetch clean markdown instead of parsing JS-rendered HTML. Also advertises the markdown alternate via `<link rel="alternate" type="text/markdown">` on marketing and docs pages so agents can discover it. Pricing is generated dynamically via `generatePricingContent()` (single source of truth with `/llms.txt` and `/llms-full.txt`); the other nine slugs are bundled at build time from `content/md/*.md` into a `MD_CONTENT` map. Supersedes #44891 (rebased fresh off current master to avoid a 9-commit replay over rename/rename conflicts created by #44897). ## Changes - New `/api-v2/md/[...slug]` route handler returns the bundled markdown (or dynamic pricing) with `Content-Type: text/markdown`, `X-Content-Type-Options: nosniff`, and appropriate cache headers - Middleware rewrites `/<slug>.md` and `Accept: text/markdown` to the API route for the `MD_PAGES` allowlist; trailing-slash variants (`/auth/`) are normalized so they resolve the same as `/auth` - Build-time codegen `scripts/generateMdContent.mjs` scans `content/md/` and emits `app/api-v2/md/content.generated.ts` exporting both `MD_CONTENT` (Map) and `MD_PAGES` (Set, incl. dynamic `pricing`). Fails the build on slug collision between `content/md/` and `DYNAMIC_SLUGS`. Adding a new marketing `.md` is just dropping a file in `content/md/` (also update `PRODUCT_OVERVIEW_LINKS` in `/llms.txt` since that list is editorial). - 8 permanent redirects `/llms/<product>.txt` → `/<product>.md` so legacy URLs in caches and downstream `llms.txt` copies keep working - `/llms.txt` product overview now references `.md` URLs (incl. `modules/cron`, `modules/queues`); `/llms-full.txt` iterates `MD_CONTENT.values()` (homepage first, then alphabetical) and appends dynamic pricing - `/llms/[slug]` route slimmed to proxy SDK reference files (`js.txt`, `dart.txt`, etc.) since redirects handle product slugs and pricing; pricing branch retained as fallback in case redirects are bypassed - `apps/www/pages/_app.tsx` injects the alternate link conditionally based on `MD_PAGES`; `/pricing` (app router) sets it via page metadata - `apps/docs/app/page.tsx` (the `/docs` root) sets the text/markdown alternate to `/llms-full.txt`; per-guide pages override with their specific `.md` URL via `genGuideMeta` in `GuidesMdx.utils.tsx`. Other docs pages (reference, troubleshooting) inherit nothing. - `apps/www/.vercelignore`: replaces the prior `*.md`/`README.md` rules with `*.md` + `!content/md/**/*.md` so Edge Function READMEs and future scratch `.md` files aren't silently shipped to the build artifact - Drops `apps/www/data/llms/*.txt` and the related `outputFileTracingIncludes` - Test coverage for the new middleware branches: `.md` suffix rewrite (allowlisted vs. fall-through), `Accept: text/markdown` content negotiation, trailing-slash normalization ## Testing (Vercel preview) Local dev server smoke tests passing on `:3771` after each iteration. Re-verified on the preview URL after the latest hardening commit: - [x] `curl -I https://<preview>/llms/auth.txt` — expect `308 Permanent Redirect` to `/auth.md` - [x] `curl https://<preview>/auth.md | head -3` — expect `# Supabase Auth` - [x] `curl https://<preview>/pricing.md | head -3` — expect `# Supabase Pricing` with current tier values - [x] `curl https://<preview>/modules/cron.md | head -3` — expect `# Supabase Cron` - [x] `curl -H 'Accept: text/markdown' https://<preview>/ | head -3` — expect `# Supabase` (homepage.md) - [x] `curl https://<preview>/llms.txt` — Product Overview section lists `.md` URLs and includes Cron + Queues - [x] `curl https://<preview>/llms-full.txt | grep -E '^# Supabase (Cron\|Queues\|Pricing)'` — Cron and Pricing each match once; Queues matches twice (marketing module + existing docs guide) - [x] View source on `/`, `/pricing`, `/database` — expect `<link rel="alternate" type="text/markdown" href="/<slug>.md">` - [x] View source on `/docs` — expect `<link rel="alternate" type="text/markdown" href="/llms-full.txt">` - [x] View source on a docs guide page (e.g., `/docs/guides/auth`) — expect per-guide `.md` alternate; reference/troubleshooting pages should NOT emit a markdown alternate - [x] `curl -I https://<preview>/auth.md` — expect `X-Content-Type-Options: nosniff` - [x] `curl -I -L -H 'Accept: text/markdown' https://<preview>/auth/` — should resolve to markdown content (trailing-slash normalization, with Vercel's auto-redirect) ## Linear - fixes GROWTH-760 ## Follow-up (separate PR) GROWTH-760 also asks about extending `.md` to blog/customers/events. Different mechanism (path-prefix middleware, MDX read at request time via `gray-matter`) so it deserves its own review. Will open a follow-up PR after this lands.  ## Summary by CodeRabbit * **New Features** * Serve prebuilt and dynamic Markdown docs via new markdown endpoints and routing; pages now advertise markdown alternates (including pricing). * Added Cron and Queues module documentation pages. * **Documentation** * Minor formatting tweaks to Realtime and Storage docs. * **Chores** * Added build-time Markdown content generation and adjusted ignore/deploy rules for generated files. * Added redirects from legacy text-based product URLs to new markdown pages. * **Tests** * Expanded tests for markdown routing and content-negotiation behavior.
2026-10-05 09:25:06 +03:00 · 2026-04-28 16:41:03 +09:00
1 parent bedb2efb87
commit d409836ca7
@@ -29,7 +29,10 @@ const generateMetadata = async (_, parent: ResolvingMetadata): Promise<Metadata>
      ...(parentAlternates && {
        languages: parentAlternates.languages || undefined,
        media: parentAlternates.media || undefined,
-        types: parentAlternates.types || undefined,
+        types: {
+          ...(parentAlternates.types ?? {}),
+          'text/markdown': '/llms-full.txt',
+        },
      }),
    },
  }
@@ -1,23 +1,23 @@
-import * as Sentry from '@sentry/nextjs'
-import { fromMarkdown } from 'mdast-util-from-markdown'
-import { gfmFromMarkdown } from 'mdast-util-gfm'
-import { gfm } from 'micromark-extension-gfm'
-import { type Metadata, type ResolvingMetadata } from 'next'
-import { notFound } from 'next/navigation'
 import { readdir } from 'node:fs/promises'
 import { extname, join, relative, sep } from 'node:path'
-
+import * as Sentry from '@sentry/nextjs'
 import { extractMessageFromAnyError, FileNotFoundError } from '~/app/api/utils'
 import { pluckPromise } from '~/features/helpers.fn'
 import { cache_fullProcess_withDevCacheBust, existsFile } from '~/features/helpers.fs'
 import type { OrPromise } from '~/features/helpers.types'
 import { generateOpenGraphImageMeta } from '~/features/seo/openGraph'
 import { BASE_PATH } from '~/lib/constants'
+import { getCustomContent } from '~/lib/custom-content/getCustomContent'
 import { GUIDES_DIRECTORY, isValidGuideFrontmatter, type GuideFrontmatter } from '~/lib/docs'
 import { GuideModelLoader } from '~/resources/guide/guideModelLoader'
+import { fromMarkdown } from 'mdast-util-from-markdown'
+import { gfmFromMarkdown } from 'mdast-util-gfm'
+import { gfm } from 'micromark-extension-gfm'
+import { type Metadata, type ResolvingMetadata } from 'next'
+import { notFound } from 'next/navigation'
+
 import { newEditLink } from './GuidesMdx.template'
 import { checkGuidePageEnabled } from './NavigationPageStatus.utils'
-import { getCustomContent } from '~/lib/custom-content/getCustomContent'

 const { metadataTitle } = getCustomContent(['metadata:title'])

@@ -179,6 +179,10 @@ const genGuideMeta =
      alternates: {
        ...parentAlternates,
        canonical: meta.canonical || `${BASE_PATH}${pathname}`,
+        types: {
+          ...(parentAlternates?.types ?? {}),
+          'text/markdown': `${BASE_PATH}${pathname}.md`,
+        },
      },
      openGraph: {
        ...parentOg,
@@ -28,6 +28,9 @@ yarn-error.log*
 # Sitemap
 public/sitemap.xml

+# Generated .md content bundle (built by scripts/generateMdContent.mjs)
+app/api-v2/md/content.generated.ts
+
 # contentlayer
 .contentlayer
 .vercel
@@ -30,8 +30,12 @@ public/fonts/**/*
 **/coverage

 # Documentation
+# We can't use a `*.md` + `!content/md/**` pattern here — Vercel's ignore
+# engine excludes content/md/ anyway, breaking generateMdContent.mjs at build
+# time. Match READMEs explicitly. Other stray .md files (Edge Function READMEs,
+# scratch notes) will ship in the build artifact but aren't routable.
 README.md
-*.md
+**/README.md
 docs/**/*

 # IDE files
@@ -0,0 +1,39 @@
+import { NextResponse } from 'next/server'
+
+import { MD_CONTENT } from '../content.generated'
+import { generatePricingContent } from '@/lib/llms'
+
+// Static .md files are bundled at build time, so they're safe to cache at the
+// edge for a day. Without s-maxage Vercel's CDN won't cache the response and
+// every request would hit the lambda.
+const STATIC_HEADERS = {
+  'Content-Type': 'text/markdown; charset=utf-8',
+  'X-Content-Type-Options': 'nosniff',
+  'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600',
+  Vary: 'Accept',
+}
+
+// Pricing is generated dynamically from shared-data without a content rebuild,
+// so use a shorter edge cache to match /llms.txt and /llms-full.txt.
+const DYNAMIC_HEADERS = {
+  'Content-Type': 'text/markdown; charset=utf-8',
+  'X-Content-Type-Options': 'nosniff',
+  'Cache-Control': 'public, s-maxage=3600, stale-while-revalidate=86400',
+  Vary: 'Accept',
+}
+
+export async function GET(_request: Request, { params }: { params: Promise<{ slug: string[] }> }) {
+  const { slug } = await params
+  const slugPath = slug.join('/')
+
+  if (slugPath === 'pricing') {
+    return new NextResponse(generatePricingContent(), { headers: DYNAMIC_HEADERS })
+  }
+
+  const content = MD_CONTENT.get(slugPath)
+  if (!content) {
+    return new NextResponse('Not found', { status: 404 })
+  }
+
+  return new NextResponse(content, { headers: STATIC_HEADERS })
+}
@@ -1,7 +1,6 @@
-import { readFile } from 'node:fs/promises'
-import { join } from 'node:path'
 import { isFeatureEnabled } from 'common/enabled-features'

+import { MD_CONTENT } from '@/app/api-v2/md/content.generated'
 import { generatePricingContent } from '@/lib/llms'

 export const dynamic = 'force-dynamic'
@@ -33,25 +32,10 @@ function getSources(): Source[] {
  ]
 }

-// Order matters: homepage first, products alphabetical in between.
-// pricing.txt is generated dynamically via generatePricingContent().
-const PRODUCT_LLM_FILES = [
-  'homepage.txt',
-  'auth.txt',
-  'database.txt',
-  'edge-functions.txt',
-  'realtime.txt',
-  'storage.txt',
-  'vector.txt',
-]
-
+// Order is set by scripts/generateMdContent.mjs (homepage first, rest
+// alphabetical). pricing is appended here since it's dynamic.
 async function readProductOverviews(): Promise<string> {
-  const staticContents = await Promise.all(
-    PRODUCT_LLM_FILES.map((file) => {
-      const filePath = join(process.cwd(), 'data/llms', file)
-      return readFile(filePath, 'utf-8')
-    })
-  )
+  const staticContents = [...MD_CONTENT.values()]
  const pricingContent = generatePricingContent()

  return [...staticContents, pricingContent].join('\n\n---\n\n')
@@ -29,15 +29,21 @@ function getSources(): Source[] {
  ]
 }

+// Editorial ordering for the product overview list (mirrors the homepage
+// products section); not derived from MD_PAGES because the order is
+// intentional. When dropping a new content/md/<slug>.md file, add a matching
+// entry here too — otherwise the page ships but won't be linked from /llms.txt.
 const PRODUCT_OVERVIEW_LINKS = [
-  '- [Supabase Overview](https://supabase.com/llms/homepage.txt)',
-  '- [Supabase Database](https://supabase.com/llms/database.txt)',
-  '- [Supabase Auth](https://supabase.com/llms/auth.txt)',
-  '- [Supabase Storage](https://supabase.com/llms/storage.txt)',
-  '- [Supabase Edge Functions](https://supabase.com/llms/edge-functions.txt)',
-  '- [Supabase Realtime](https://supabase.com/llms/realtime.txt)',
-  '- [Supabase Vector](https://supabase.com/llms/vector.txt)',
-  '- [Supabase Pricing](https://supabase.com/llms/pricing.txt)',
+  '- [Supabase Overview](https://supabase.com/homepage.md)',
+  '- [Supabase Database](https://supabase.com/database.md)',
+  '- [Supabase Auth](https://supabase.com/auth.md)',
+  '- [Supabase Storage](https://supabase.com/storage.md)',
+  '- [Supabase Edge Functions](https://supabase.com/edge-functions.md)',
+  '- [Supabase Realtime](https://supabase.com/realtime.md)',
+  '- [Supabase Vector](https://supabase.com/vector.md)',
+  '- [Supabase Cron](https://supabase.com/modules/cron.md)',
+  '- [Supabase Queues](https://supabase.com/modules/queues.md)',
+  '- [Supabase Pricing](https://supabase.com/pricing.md)',
 ].join('\n')

 export async function GET() {
@@ -1,6 +1,3 @@
-import { readFile } from 'node:fs/promises'
-import { join } from 'node:path'
-
 import { generatePricingContent } from '@/lib/llms'

 export const dynamic = 'force-dynamic'
@@ -14,6 +11,10 @@ function textResponse(content: string) {
  })
 }

+// Product overview slugs and pricing.txt are 301-redirected to /<slug>.md in
+// apps/www/lib/redirects.js, so this handler typically only sees per-SDK
+// reference files. The pricing branch stays as a fallback in case the redirect
+// is bypassed.
 export async function GET(_request: Request, { params }: { params: Promise<{ slug: string }> }) {
  const { slug } = await params

@@ -21,21 +22,10 @@ export async function GET(_request: Request, { params }: { params: Promise<{ slu
    return new Response('Not Found', { status: 404 })
  }

-  // 1. Check for dynamically generated content (e.g. pricing)
  if (slug === 'pricing.txt') {
    return textResponse(generatePricingContent())
  }

-  // 2. Check for a local static file in public/llms/
-  try {
-    const filePath = join(process.cwd(), 'data/llms', slug)
-    const content = await readFile(filePath, 'utf-8')
-    return textResponse(content)
-  } catch {
-    // File doesn't exist locally, fall through
-  }
-
-  // 3. Try fetching from the docs app
  const docsUrl = process.env.NEXT_PUBLIC_DOCS_URL
  if (docsUrl) {
    const response = await fetch(`${docsUrl}/llms/${slug}`)
@@ -1,10 +1,16 @@
 import type { Metadata } from 'next'
+
 import PricingContent from './PricingContent'

 export const metadata: Metadata = {
  title: 'Pricing & Fees | Supabase',
  description:
    'Explore Supabase fees and pricing information. Find our competitive pricing Plans, with no hidden pricing. We have a generous Free Plan for those getting started, and Pay As You Go for those scaling up.',
+  alternates: {
+    types: {
+      'text/markdown': '/pricing.md',
+    },
+  },
  openGraph: {
    title: 'Pricing & Fees | Supabase',
    description:
@@ -0,0 +1,38 @@
+# Supabase Cron
+
+> Schedule and manage recurring jobs directly in Postgres with pg_cron.
+
+Supabase Cron is a Postgres module that uses the pg_cron extension to schedule and manage recurring jobs. Define schedules with standard cron syntax or natural language, and run jobs that call database functions, Edge Functions, or remote webhooks.
+
+## Key Features
+
+- **Postgres native**: schedule and run jobs directly within your database, no external scheduler needed
+- **Cron syntax and natural language**: use familiar cron expressions or plain English to define intervals
+- **Sub-minute scheduling**: run jobs as frequently as every 1-59 seconds
+- **Real-time monitoring**: track and debug scheduled jobs with built-in observability tools
+- **Extensible**: trigger database functions, Supabase Edge Functions, or HTTP webhooks
+- **Dashboard management**: create, edit, and monitor jobs through an intuitive UI
+- **SQL-based**: manage jobs using simple SQL commands, track changes with Postgres migrations
+- **100% open source**: built on pg_cron, a trusted community-driven extension
+
+## Common Use Cases
+
+- Periodic data cleanup or archival
+- Scheduled report generation
+- Recurring API calls or webhook triggers
+- Database maintenance tasks (vacuum, reindex)
+- Timed cache invalidation
+- Periodic data synchronization between systems
+
+## Technical Details
+
+- Extension: pg_cron (open source)
+- Minimum interval: 1 second
+- Schedule format: cron syntax (minute/hour/day/month/weekday) or natural language
+- Job targets: SQL statements, database functions, Edge Functions, HTTP endpoints
+- Monitoring: job run history with status, duration, and error details
+
+## Links
+
+- Documentation: https://supabase.com/docs/guides/cron
+- Dashboard: https://supabase.com/dashboard/project/_/integrations/cron/overview
@@ -0,0 +1,37 @@
+# Supabase Queues
+
+> Durable message queues with guaranteed delivery, powered by Postgres and pgmq.
+
+Supabase Queues is a Postgres module that uses the pgmq extension to provide durable message queues with exactly-once delivery within a visibility window. Manage queues using SQL, the Supabase client libraries, or the Dashboard.
+
+## Key Features
+
+- **Postgres native**: create and manage queues directly within your database
+- **Exactly-once delivery**: messages are delivered exactly once within a configurable visibility window
+- **Message archival**: archive messages instead of deleting them for audit trails and future reference
+- **Real-time monitoring**: track and manage messages with built-in observability tools
+- **Multiple access methods**: manage via SQL, PostgREST API (server-side or client-side), or Dashboard
+- **Dashboard management**: create queues, send messages, and monitor processing in real time
+- **100% open source**: built on pgmq, a trusted community-driven extension
+
+## Common Use Cases
+
+- Asynchronous task processing
+- Event-driven workflows
+- Background job scheduling
+- Inter-service communication
+- Webhook delivery with retry logic
+- Order processing and fulfillment pipelines
+
+## Technical Details
+
+- Extension: pgmq (open source)
+- Delivery guarantee: exactly-once within visibility window
+- Message format: JSONB payload
+- Access methods: SQL, PostgREST API, Supabase client libraries
+- Monitoring: queue depth, message status, processing metrics
+
+## Links
+
+- Documentation: https://supabase.com/docs/guides/queues
+- Dashboard: https://supabase.com/dashboard/project/_/integrations/queues/overview
@@ -7,12 +7,15 @@ Supabase Realtime enables live data synchronization between your database and co
 ## Capabilities

 ### Database Changes
+
 Listen to Postgres INSERT, UPDATE, and DELETE events in real time. Subscribe to specific tables, filter by columns, and receive only the changes you care about. Powered by Postgres logical replication.

 ### Presence
+
 Store and synchronize online user state across all connected clients. Track who is online, what page they are viewing, or their cursor position. State is automatically cleaned up when clients disconnect.

 ### Broadcast
+
 Send arbitrary messages to all clients subscribed to the same Channel. Useful for typing indicators, live cursors, game state, notifications, or any real-time communication that does not need to be persisted.

 ## Technical Details
@@ -15,12 +15,15 @@ Supabase Storage is an open source object store that integrates natively with Su
 ## Bucket Types

 ### Files Buckets
+
 For everyday assets and user content: images, videos, documents, PDFs, archives. Served from the global CDN with fine-grained access controls via RLS policies.

 ### Analytics Buckets
+
 For large-scale analytical workloads on open table formats (Apache Iceberg). Designed for historical data, time-series data, logs, and ETL outputs. Optionally queryable via Postgres.

 ### Vector Buckets
+
 For AI/ML workloads. Store and index vector embeddings with multiple distance metrics, metadata filtering, and fast similarity queries for RAG systems and AI-powered search.

 ## Technical Details
@@ -3127,4 +3127,13 @@ module.exports = [
    source: '/docs/llms-full.txt',
    destination: '/llms-full.txt',
  },
+  // Legacy product .txt URLs → new .md routes
+  { permanent: true, source: '/llms/homepage.txt', destination: '/homepage.md' },
+  { permanent: true, source: '/llms/auth.txt', destination: '/auth.md' },
+  { permanent: true, source: '/llms/database.txt', destination: '/database.md' },
+  { permanent: true, source: '/llms/edge-functions.txt', destination: '/edge-functions.md' },
+  { permanent: true, source: '/llms/realtime.txt', destination: '/realtime.md' },
+  { permanent: true, source: '/llms/storage.txt', destination: '/storage.md' },
+  { permanent: true, source: '/llms/vector.txt', destination: '/vector.md' },
+  { permanent: true, source: '/llms/pricing.txt', destination: '/pricing.md' },
 ]
@@ -1,17 +1,26 @@
-import { NextRequest } from 'next/server'
-import { describe, expect, it } from 'vitest'
-
 import { FIRST_REFERRER_COOKIE_NAME } from 'common/first-referrer-cookie'
+import { NextRequest } from 'next/server'
+import { describe, expect, it, vi } from 'vitest'

 import { middleware } from './middleware'

+// content.generated.ts is produced by scripts/generateMdContent.mjs at
+// content:build time and gitignored, so it isn't on disk in CI before tests
+// run. The mock seeds a representative allowlist so the .md-routing branches
+// are actually exercised below.
+vi.mock('./app/api-v2/md/content.generated', () => ({
+  MD_CONTENT: new Map<string, string>(),
+  MD_PAGES: new Set<string>(['homepage', 'auth', 'pricing']),
+}))
+
 function makeRequest(
  url: string,
-  { referer, hasCookie }: { referer?: string; hasCookie?: boolean } = {}
+  { referer, hasCookie, accept }: { referer?: string; hasCookie?: boolean; accept?: string } = {}
 ): NextRequest {
-  const req = new NextRequest(new URL(url, 'https://supabase.com'), {
-    headers: referer ? { referer } : {},
-  })
+  const headers: Record<string, string> = {}
+  if (referer) headers.referer = referer
+  if (accept) headers.accept = accept
+  const req = new NextRequest(new URL(url, 'https://supabase.com'), { headers })
  if (hasCookie) {
    req.cookies.set(FIRST_REFERRER_COOKIE_NAME, 'existing')
  }
@@ -84,4 +93,59 @@ describe('www middleware', () => {
      expect(res.cookies.get(FIRST_REFERRER_COOKIE_NAME)).toBeUndefined()
    })
  })
+
+  describe('.md suffix routing', () => {
+    it('rewrites /<slug>.md for allowlisted slugs', () => {
+      const req = makeRequest('/auth.md')
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBe('https://supabase.com/api-v2/md/auth')
+    })
+
+    it('falls through for non-allowlisted .md slugs', () => {
+      const req = makeRequest('/not-a-page.md')
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBeNull()
+    })
+  })
+
+  describe('Accept: text/markdown content negotiation', () => {
+    it('rewrites / to homepage when Accept: text/markdown', () => {
+      const req = makeRequest('/', { accept: 'text/markdown' })
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBe(
+        'https://supabase.com/api-v2/md/homepage'
+      )
+    })
+
+    it('rewrites /<slug> when Accept: text/markdown matches the allowlist', () => {
+      const req = makeRequest('/auth', { accept: 'text/markdown' })
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBe('https://supabase.com/api-v2/md/auth')
+    })
+
+    it('rewrites /<slug>/ (trailing slash) the same as /<slug>', () => {
+      const req = makeRequest('/auth/', { accept: 'text/markdown' })
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBe('https://supabase.com/api-v2/md/auth')
+    })
+
+    it('falls through when Accept does not include text/markdown', () => {
+      const req = makeRequest('/auth', { accept: 'text/html' })
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBeNull()
+    })
+
+    it('falls through when slug is not in the allowlist', () => {
+      const req = makeRequest('/not-a-page', { accept: 'text/markdown' })
+      const res = middleware(req)
+
+      expect(res.headers.get('x-middleware-rewrite')).toBeNull()
+    })
+  })
 })
@@ -1,7 +1,31 @@
 import { stampFirstReferrerCookie } from 'common/first-referrer-cookie'
 import { NextResponse, type NextRequest } from 'next/server'

+import { MD_PAGES } from './app/api-v2/md/content.generated'
+
 export function middleware(request: NextRequest) {
+  const { pathname } = request.nextUrl
+
+  // Handle /<page>.md suffix: /pricing.md -> /api-v2/md/pricing
+  if (pathname.endsWith('.md')) {
+    const slug = pathname.slice(1, -3) // strip leading / and trailing .md
+    if (MD_PAGES.has(slug)) {
+      return NextResponse.rewrite(new URL(`/api-v2/md/${slug}`, request.nextUrl))
+    }
+  }
+
+  // Content negotiation: Accept: text/markdown on known pages
+  const accept = (request.headers.get('accept') ?? '').toLowerCase()
+  if (accept.includes('text/markdown')) {
+    // Strip trailing slash so /auth/ and /auth resolve to the same allowlist entry.
+    // (NextURL's pathname setter preserves the trailing-slash style of the cloned
+    // origin, which would otherwise leak through to the rewrite target.)
+    const slug = (pathname === '/' ? 'homepage' : pathname.slice(1)).replace(/\/$/, '')
+    if (MD_PAGES.has(slug)) {
+      return NextResponse.rewrite(new URL(`/api-v2/md/${slug}`, request.nextUrl))
+    }
+  }
+
  const response = NextResponse.next()
  stampFirstReferrerCookie(request, response)
  return response
@@ -71,10 +71,6 @@ const nextConfig = {
      'public/**/*',
    ],
  },
-  outputFileTracingIncludes: {
-    '/llms-full.txt': ['./data/llms/**/*'],
-    '/llms/[slug]': ['./data/llms/**/*'],
-  },
  reactStrictMode: true,
  images: {
    dangerouslyAllowSVG: false,
@@ -13,7 +13,7 @@
    "clean": "rimraf node_modules",
    "pretypecheck": "next typegen",
    "typecheck": "pnpm run content:build && tsc --noEmit",
-    "content:build": "node scripts/generateStaticContent.mjs",
+    "content:build": "node scripts/generateStaticContent.mjs && node scripts/generateMdContent.mjs",
    "test": "vitest --run",
    "test:watch": "vitest watch",
    "postbuild": "node ./internals/generate-sitemap.mjs && ./../../scripts/upload-static-assets.sh"
@@ -26,6 +26,7 @@ import { useConsentToast } from 'ui-patterns/consent'

 import useDarkLaunchWeeks from '../hooks/useDarkLaunchWeeks'
 import { useWwwCommandMenuTelemetry } from '../hooks/useWwwCommandMenuTelemetry'
+import { MD_PAGES } from '@/app/api-v2/md/content.generated'
 import { Toaster } from '@/app/toaster'
 import { WwwCommandMenu } from '@/components/CommandMenu'
 import { API_URL, APP_NAME, DEFAULT_META_DESCRIPTION } from '@/lib/constants'
@@ -53,10 +54,20 @@ export default function App({ Component, pageProps }: AppProps) {
    themeColor = 'FFFFFF'
  }

+  // Advertise the .md version for AI agents on pages that have one.
+  const cleanPath = (router.asPath ?? '/').split('?')[0].split('#')[0].replace(/\/$/, '') || '/'
+  const mdSlug = cleanPath === '/' ? 'homepage' : cleanPath.slice(1)
+  const mdAlternateHref = MD_PAGES.has(mdSlug)
+    ? cleanPath === '/'
+      ? '/homepage.md'
+      : `${cleanPath}.md`
+    : null
+
  return (
    <>
      <Head>
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+        {mdAlternateHref && <link rel="alternate" type="text/markdown" href={mdAlternateHref} />}
      </Head>
      <MetaFaviconsPagesRouter
        applicationName={applicationName}
@@ -0,0 +1,98 @@
+// @ts-check
+
+/**
+ * Scans content/md/ and emits a TypeScript module exporting the markdown
+ * content as a Map plus the slug allowlist as a Set. Static imports of the
+ * generated file make the content traceable by @vercel/nft so it ends up in
+ * the serverless bundle without runtime fs.readFile.
+ *
+ * Drop a new .md file in content/md/ and the route handler picks it up on
+ * the next content:build — no constant to edit.
+ */
+
+import { promises as fs } from 'fs'
+import path from 'path'
+import { fileURLToPath } from 'url'
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url))
+const contentDir = path.join(__dirname, '../content/md')
+const outputPath = path.join(__dirname, '../app/api-v2/md/content.generated.ts')
+
+// 'pricing' is served dynamically via generatePricingContent() instead of
+// from a .md file; it still needs to be in MD_PAGES so middleware rewrites
+// /pricing.md to the API route.
+const DYNAMIC_SLUGS = ['pricing']
+
+async function collectMdFiles(dir, prefix = '') {
+  const results = []
+  const dirents = await fs.readdir(dir, { withFileTypes: true })
+  for (const dirent of dirents) {
+    const slug = prefix ? `${prefix}/${dirent.name}` : dirent.name
+    if (dirent.isDirectory()) {
+      results.push(...(await collectMdFiles(path.join(dir, dirent.name), slug)))
+    } else if (dirent.name.endsWith('.md')) {
+      results.push(slug.replace(/\.md$/, ''))
+    }
+  }
+  return results
+}
+
+// homepage first (it's the site overview), then everything else alphabetical.
+function sortSlugs(a, b) {
+  if (a === 'homepage') return -1
+  if (b === 'homepage') return 1
+  return a.localeCompare(b)
+}
+
+const slugs = (await collectMdFiles(contentDir)).sort(sortSlugs)
+
+if (slugs.length === 0) {
+  console.error('❌ No .md files found in content/md/')
+  process.exit(1)
+}
+
+// A static file with the same slug as a dynamic generator would land in both
+// MD_CONTENT and the dynamic append path, so /llms-full.txt would emit it
+// twice. Fail the build instead of shipping a duplicate.
+const collisions = slugs.filter((s) => DYNAMIC_SLUGS.includes(s))
+if (collisions.length > 0) {
+  console.error(
+    `❌ Slug collision: [${collisions.join(', ')}] is reserved for a dynamic generator. ` +
+      `Remove the corresponding file from content/md/ or update DYNAMIC_SLUGS.`
+  )
+  process.exit(1)
+}
+
+const entries = []
+const errors = []
+for (const slug of slugs) {
+  const filePath = path.join(contentDir, `${slug}.md`)
+  try {
+    const content = await fs.readFile(filePath, 'utf-8')
+    entries.push(`  [${JSON.stringify(slug)}, ${JSON.stringify(content)}]`)
+  } catch (err) {
+    errors.push(`${slug}.md: ${err.message}`)
+  }
+}
+
+if (errors.length > 0) {
+  console.error('❌ Failed to read .md files:')
+  errors.forEach((e) => console.error(`   ${e}`))
+  process.exit(1)
+}
+
+const allSlugs = [...slugs, ...DYNAMIC_SLUGS]
+const pageEntries = allSlugs.map((s) => `  ${JSON.stringify(s)}`).join(',\n')
+
+const output = `// AUTO-GENERATED by scripts/generateMdContent.mjs — do not edit
+export const MD_CONTENT = new Map<string, string>([
+${entries.join(',\n')},
+])
+
+export const MD_PAGES = new Set<string>([
+${pageEntries},
+])
+`
+
+await fs.writeFile(outputPath, output, 'utf-8')
+console.log(`✅ Generated ${outputPath} (${entries.length} files, ${allSlugs.length} pages)`)