mirror of
https://github.com/supabase/supabase.git
synced 2026-10-11 20:35:07 +03:00
## Summary Adds `/<page>.md` routes for 10 marketing/product pages (homepage, auth, database, edge-functions, realtime, storage, vector, pricing, modules/cron, modules/queues) so AI agents can fetch clean markdown instead of parsing JS-rendered HTML. Also advertises the markdown alternate via `<link rel="alternate" type="text/markdown">` on marketing and docs pages so agents can discover it. Pricing is generated dynamically via `generatePricingContent()` (single source of truth with `/llms.txt` and `/llms-full.txt`); the other nine slugs are bundled at build time from `content/md/*.md` into a `MD_CONTENT` map. Supersedes #44891 (rebased fresh off current master to avoid a 9-commit replay over rename/rename conflicts created by #44897). ## Changes - New `/api-v2/md/[...slug]` route handler returns the bundled markdown (or dynamic pricing) with `Content-Type: text/markdown`, `X-Content-Type-Options: nosniff`, and appropriate cache headers - Middleware rewrites `/<slug>.md` and `Accept: text/markdown` to the API route for the `MD_PAGES` allowlist; trailing-slash variants (`/auth/`) are normalized so they resolve the same as `/auth` - Build-time codegen `scripts/generateMdContent.mjs` scans `content/md/` and emits `app/api-v2/md/content.generated.ts` exporting both `MD_CONTENT` (Map) and `MD_PAGES` (Set, incl. dynamic `pricing`). Fails the build on slug collision between `content/md/` and `DYNAMIC_SLUGS`. Adding a new marketing `.md` is just dropping a file in `content/md/` (also update `PRODUCT_OVERVIEW_LINKS` in `/llms.txt` since that list is editorial). - 8 permanent redirects `/llms/<product>.txt` → `/<product>.md` so legacy URLs in caches and downstream `llms.txt` copies keep working - `/llms.txt` product overview now references `.md` URLs (incl. `modules/cron`, `modules/queues`); `/llms-full.txt` iterates `MD_CONTENT.values()` (homepage first, then alphabetical) and appends dynamic pricing - `/llms/[slug]` route slimmed to proxy SDK reference files (`js.txt`, `dart.txt`, etc.) since redirects handle product slugs and pricing; pricing branch retained as fallback in case redirects are bypassed - `apps/www/pages/_app.tsx` injects the alternate link conditionally based on `MD_PAGES`; `/pricing` (app router) sets it via page metadata - `apps/docs/app/page.tsx` (the `/docs` root) sets the text/markdown alternate to `/llms-full.txt`; per-guide pages override with their specific `.md` URL via `genGuideMeta` in `GuidesMdx.utils.tsx`. Other docs pages (reference, troubleshooting) inherit nothing. - `apps/www/.vercelignore`: replaces the prior `*.md`/`README.md` rules with `*.md` + `!content/md/**/*.md` so Edge Function READMEs and future scratch `.md` files aren't silently shipped to the build artifact - Drops `apps/www/data/llms/*.txt` and the related `outputFileTracingIncludes` - Test coverage for the new middleware branches: `.md` suffix rewrite (allowlisted vs. fall-through), `Accept: text/markdown` content negotiation, trailing-slash normalization ## Testing (Vercel preview) Local dev server smoke tests passing on `:3771` after each iteration. Re-verified on the preview URL after the latest hardening commit: - [x] `curl -I https://<preview>/llms/auth.txt` — expect `308 Permanent Redirect` to `/auth.md` - [x] `curl https://<preview>/auth.md | head -3` — expect `# Supabase Auth` - [x] `curl https://<preview>/pricing.md | head -3` — expect `# Supabase Pricing` with current tier values - [x] `curl https://<preview>/modules/cron.md | head -3` — expect `# Supabase Cron` - [x] `curl -H 'Accept: text/markdown' https://<preview>/ | head -3` — expect `# Supabase` (homepage.md) - [x] `curl https://<preview>/llms.txt` — Product Overview section lists `.md` URLs and includes Cron + Queues - [x] `curl https://<preview>/llms-full.txt | grep -E '^# Supabase (Cron\|Queues\|Pricing)'` — Cron and Pricing each match once; Queues matches twice (marketing module + existing docs guide) - [x] View source on `/`, `/pricing`, `/database` — expect `<link rel="alternate" type="text/markdown" href="/<slug>.md">` - [x] View source on `/docs` — expect `<link rel="alternate" type="text/markdown" href="/llms-full.txt">` - [x] View source on a docs guide page (e.g., `/docs/guides/auth`) — expect per-guide `.md` alternate; reference/troubleshooting pages should NOT emit a markdown alternate - [x] `curl -I https://<preview>/auth.md` — expect `X-Content-Type-Options: nosniff` - [x] `curl -I -L -H 'Accept: text/markdown' https://<preview>/auth/` — should resolve to markdown content (trailing-slash normalization, with Vercel's auto-redirect) ## Linear - fixes GROWTH-760 ## Follow-up (separate PR) GROWTH-760 also asks about extending `.md` to blog/customers/events. Different mechanism (path-prefix middleware, MDX read at request time via `gray-matter`) so it deserves its own review. Will open a follow-up PR after this lands. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Serve prebuilt and dynamic Markdown docs via new markdown endpoints and routing; pages now advertise markdown alternates (including pricing). * Added Cron and Queues module documentation pages. * **Documentation** * Minor formatting tweaks to Realtime and Storage docs. * **Chores** * Added build-time Markdown content generation and adjusted ignore/deploy rules for generated files. * Added redirects from legacy text-based product URLs to new markdown pages. * **Tests** * Expanded tests for markdown routing and content-negotiation behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
99 lines
3.2 KiB
JavaScript
99 lines
3.2 KiB
JavaScript
// @ts-check
|
|
|
|
/**
|
|
* Scans content/md/ and emits a TypeScript module exporting the markdown
|
|
* content as a Map plus the slug allowlist as a Set. Static imports of the
|
|
* generated file make the content traceable by @vercel/nft so it ends up in
|
|
* the serverless bundle without runtime fs.readFile.
|
|
*
|
|
* Drop a new .md file in content/md/ and the route handler picks it up on
|
|
* the next content:build — no constant to edit.
|
|
*/
|
|
|
|
import { promises as fs } from 'fs'
|
|
import path from 'path'
|
|
import { fileURLToPath } from 'url'
|
|
|
|
const __dirname = path.dirname(fileURLToPath(import.meta.url))
|
|
const contentDir = path.join(__dirname, '../content/md')
|
|
const outputPath = path.join(__dirname, '../app/api-v2/md/content.generated.ts')
|
|
|
|
// 'pricing' is served dynamically via generatePricingContent() instead of
|
|
// from a .md file; it still needs to be in MD_PAGES so middleware rewrites
|
|
// /pricing.md to the API route.
|
|
const DYNAMIC_SLUGS = ['pricing']
|
|
|
|
async function collectMdFiles(dir, prefix = '') {
|
|
const results = []
|
|
const dirents = await fs.readdir(dir, { withFileTypes: true })
|
|
for (const dirent of dirents) {
|
|
const slug = prefix ? `${prefix}/${dirent.name}` : dirent.name
|
|
if (dirent.isDirectory()) {
|
|
results.push(...(await collectMdFiles(path.join(dir, dirent.name), slug)))
|
|
} else if (dirent.name.endsWith('.md')) {
|
|
results.push(slug.replace(/\.md$/, ''))
|
|
}
|
|
}
|
|
return results
|
|
}
|
|
|
|
// homepage first (it's the site overview), then everything else alphabetical.
|
|
function sortSlugs(a, b) {
|
|
if (a === 'homepage') return -1
|
|
if (b === 'homepage') return 1
|
|
return a.localeCompare(b)
|
|
}
|
|
|
|
const slugs = (await collectMdFiles(contentDir)).sort(sortSlugs)
|
|
|
|
if (slugs.length === 0) {
|
|
console.error('❌ No .md files found in content/md/')
|
|
process.exit(1)
|
|
}
|
|
|
|
// A static file with the same slug as a dynamic generator would land in both
|
|
// MD_CONTENT and the dynamic append path, so /llms-full.txt would emit it
|
|
// twice. Fail the build instead of shipping a duplicate.
|
|
const collisions = slugs.filter((s) => DYNAMIC_SLUGS.includes(s))
|
|
if (collisions.length > 0) {
|
|
console.error(
|
|
`❌ Slug collision: [${collisions.join(', ')}] is reserved for a dynamic generator. ` +
|
|
`Remove the corresponding file from content/md/ or update DYNAMIC_SLUGS.`
|
|
)
|
|
process.exit(1)
|
|
}
|
|
|
|
const entries = []
|
|
const errors = []
|
|
for (const slug of slugs) {
|
|
const filePath = path.join(contentDir, `${slug}.md`)
|
|
try {
|
|
const content = await fs.readFile(filePath, 'utf-8')
|
|
entries.push(` [${JSON.stringify(slug)}, ${JSON.stringify(content)}]`)
|
|
} catch (err) {
|
|
errors.push(`${slug}.md: ${err.message}`)
|
|
}
|
|
}
|
|
|
|
if (errors.length > 0) {
|
|
console.error('❌ Failed to read .md files:')
|
|
errors.forEach((e) => console.error(` ${e}`))
|
|
process.exit(1)
|
|
}
|
|
|
|
const allSlugs = [...slugs, ...DYNAMIC_SLUGS]
|
|
const pageEntries = allSlugs.map((s) => ` ${JSON.stringify(s)}`).join(',\n')
|
|
|
|
const output = `// AUTO-GENERATED by scripts/generateMdContent.mjs — do not edit
|
|
export const MD_CONTENT = new Map<string, string>([
|
|
${entries.join(',\n')},
|
|
])
|
|
|
|
export const MD_PAGES = new Set<string>([
|
|
${pageEntries},
|
|
])
|
|
`
|
|
|
|
await fs.writeFile(outputPath, output, 'utf-8')
|
|
console.log(`✅ Generated ${outputPath} (${entries.length} files, ${allSlugs.length} pages)`)
|