# Before
Crawlers pages that used MDX components with defined attributes errored.
Example: /docs/reference/kotlin/installing
# After
The pages do not error.
* Bump all versions of postcss to 8.5.3.
* Run next/codemod on the docs app.
* Move two experimental flags into stable. Add next-mdx-remote as a transpiled package.
* Add extra folders to the clean command in docs.
* Fix type errors in docs test.
* Run prettier on the new files.
* remove turbopack, fix fetch revalidation, fix metadata awaits
Couple of minor fixes:
- Turbopack doesn't work in dev because of known MDX loader limitations
(cannot load functions in MDX plugin config)
- Fetches not cached by default anymore in Next 15 so need to manually
cache the ones we need
- Missing a few awaits for metadata generation with page params
* Bump the graphiql version because headlessui/react is not building with Next 15.
---------
Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
* feat(content api): add client library api reference search results
Allow searchDocs results to also return function references from the
client library APIs
* fix(content api): refine language enum handling
Add a top-level field to search docs globally. Right now this only
returns Markdown guides (not references, GitHub discussions, or partner
pages.
The full GraphQL schema at this point:
```
schema {
query: RootQueryType
}
type RootQueryType {
"""Get the GraphQL schema for this endpoint"""
schema: String!
"""Search the Supabase docs for content matching a query string"""
searchDocs(query: String!, limit: Int): SearchResultCollection
}
"""A collection of search results containing content from Supabase docs"""
type SearchResultCollection {
"""A list of edges containing nodes in this collection"""
edges: [SearchResultEdge!]!
"""The nodes in this collection, directly accessible"""
nodes: [SearchResult!]!
"""The total count of items available in this collection"""
totalCount: Int!
}
"""An edge in a collection of SearchResults"""
type SearchResultEdge {
"""The SearchResult at the end of the edge"""
node: SearchResult!
}
"""Document that matches a search query"""
interface SearchResult {
"""The title of the matching result"""
title: String
"""The URL of the matching result"""
href: String
"""The full content of the matching result"""
content: String
}
```
Towards DOCS-214
* feat (content api): add top-level schema query
Adds a top-level schema query that returns the stringified GraphQL
schema for this endpoint. Useful for LLMs to access the stringified
schema for prompting and tool calls.
Query shape:
```
query Search {
schema
}
```
Query response:
```
{
"data": {
"schema": "schema {\n query: RootQueryType\n}\n\ntype RootQueryType
{\n \"\"\"Get the GraphQL schema for this endpoint\"\"\"\n schema:
String\n}"
}
}
```
Towards DOCS-214
* test (content api): add schema snapshot
Add schema snapshot test to prevent unintentional changes to public API
for this endpoint
* fixup! feat (content api): add top-level schema query
* fix (content api): local dev introspection
The validation rules that are used for security in production are
preventing GraphiQL from introspecting the schema in dev. Since this DX
feature is incredibly useful, I'm turning off the complexity and depth
validations for dev GraphiQL only. Local curls, etc., will still have
production-standard validation so it can be tested locally.
GraphQL errors were occasionally returned as normal HTTP status code
errors, switched to using only GraphQL-spec compliant errors for the
/docs/api/graphql endpoint.
Continue building out the docs/api/graphql endpoint by adding more
validation:
-A query depth limiter
- A query complexity limiter
Also added dummy (empty) root resolver and schema, plus a dev-only
GraphiQL page for better DX.
Endpoint should still return a 404 for now as there is no resolver.
The first PR in a series to build out the Content API. Adds a dummy
GraphQL route with some tests: for now all this does is validate the
request body and return a 404.
Before:
Revalidation was turned off entirely for wrappers pages, and done daily for GraphQL pages.
After:
Revalidation is via tag. There is an /api/revalidate endpoint that is used by GitHub Actions on the federated repos to trigger the revalidation. This decreases the number of unnecessary page generations by Vercel, and gives the maintainers of federated repos more fine-grained control over when to reploy their docs (see the docs deployment playbook for more info).
Before:
There was a bug with middleware rewrites for reference pages without
crawler versions (basically all references except the client SDKs).
There was a branch that was not handled (the requesting user agent is a
a bot, but the reference is not a client SDK), so it failed through to
the base case of NextResponse.next(), which 404s as that page does not
exist for a path like /reference/api/v1-deploy-a-function
After:
Fixed the branch. If all of the following are true:
- Is a reference page
- Requesting user agent is a bot
- Reference is not a client SDK
Then it should follow the same path as a non-bot request, because there
is no bot-specific version. That means it should be rewritten to teh
slugless page: /reference/api/v1-deploya-function => /reference/api
Add a route for manually revalidating cache contents by tag.
The route is protected by authentication to prevent abuse. Automated
actions in CI should be set up with a basic API key, which has a rate
limit of 6 hours between changes. Overriding is possible with an
override key, which should be used as an escape hatch.
Usage:
- API key provided in header `Authorization: Bearer <KEY>`
- Body has shape `{ tags: string[] }`
The end of the Great Migration is here!!!
Moves the last pages over, deleting pages like the FAQ that we don't use
and that contain duplicated information anyway.
Dev secret auth page URL had to change as App Router doesn't like the
leading underscores in the path.
Also fixes the not-found recommendations to use the proper Next.js
not-found page so it will return a 404 as it should. (I was under the
erroneous impression that I couldn't get the pathname in not-found.tsx,
that is not true, so this works better.)
Migrates client SDK References to App Router. (Management and CLI API references aren't migrated yet, nor are self-hosting config references.)
Some notes:
Big changes to the way crawler pages are built and individual section URLs (e.g., javascript/select) are served. All of these used to be SSG-generated pages, but the number of heavy pages was just too much to handle -- slow as molasses and my laptop sounded like it was taking off, and CI sometimes refuses to build it all at all.
Tried various tricks with caching and pre-generating data but no dice.
So I changed to only building one copy of each SDK+version page, then serving the sub-URLs through a response rewrite. That's for the actual user-visible pages.
For the bot pages, each sub-URL needs to be its own page, but prebuilding it doesn't work, and rendering on demand from React components is too slow (looking for super-fast response here for SEO). Instead I changed to using an API route that serves very minimal, hand-crafted HTML. It looks ugly, but it's purely for the search bots.
You can test what bots see by running curl --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" <URL_OF_PAGE>
Also added some smoke tests to run against prod for the crawler routes, since we don't keep an eye on those regularly, and Vercel config changes could surprise-break them. Tested the meta images on Open Graph and all seems to work fine.
With this approach, full production builds are really fast: ~5 minutes
Starts using the new type spec handling, which is better at finding params automatically, so I could remove some of the manually written ones from the spec files.
Charis