# Second try of making a new better process for SDK automation
Instead of building a new pipeline. We will take the lessons learned
form round 1, plus the good design and improvement on DX quality for
drop-in file as a single step required from SDK team and produce almost
identical set of files as used right now to render using the current
pipeline.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* New reference-content pipeline producing per-library reference
artifacts and integrating into prebuilds, search ingestion, and
rendering (type-aware examples).
* **Documentation**
* Added comprehensive JavaScript SDK v2 reference content and partials
(Auth MFA, passkeys, admin, TypeScript support, filters, modifiers,
Installing, Initializing, Buckets, etc.).
* **Tests & CI**
* Added regression snapshot test and updated workflows to refresh
reference snapshots and ensure spec downloads.
* **Chores**
* Updated ignore rules, build scripts, Makefile targets, and package
lifecycle hooks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Katerina Skroumpelou <mandarini@users.noreply.github.com>
Co-authored-by: Katerina Skroumpelou <sk.katherine@gmail.com>
strictNullChecks was off for docs, which lets errors slip through and
leads to incorrect required/optional typing on Zod-inferred types. This
PR enables strictNullChecks and fixes all the existing violations.
* Bump all versions of postcss to 8.5.3.
* Run next/codemod on the docs app.
* Move two experimental flags into stable. Add next-mdx-remote as a transpiled package.
* Add extra folders to the clean command in docs.
* Fix type errors in docs test.
* Run prettier on the new files.
* remove turbopack, fix fetch revalidation, fix metadata awaits
Couple of minor fixes:
- Turbopack doesn't work in dev because of known MDX loader limitations
(cannot load functions in MDX plugin config)
- Fetches not cached by default anymore in Next 15 so need to manually
cache the ones we need
- Missing a few awaits for metadata generation with page params
* Bump the graphiql version because headlessui/react is not building with Next 15.
---------
Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Braces must be escaped in non-code-blocks (or the site won't build),
but preserved in code blocks. This requires a more complex approach than
we were using:
- Parse the initial markdown and identify code blocks
- Escape braces only in non-code sections of the markdown
* chore(docs,refs): prebuild self-hosted ref data
Pregenerate self-hosted ref data files, in the style of the client lib,
Management API, and CLI references. Preparatory step for migrating the
self-hosted references to App Router.
* refactor(docs): convert self-hosting to app router
Convert self-hosting reference pages to App Router.
* fix(docs,sitemap)
* chore(docs): preprocess management api specs
In preparation for App Router migration of Management API reference,
preprocess the Management API spec (both OpenAPI spec and common
sections), and write the processed data to file, in the style of the
other pregenerated reference data.
Details:
- Split reference section generation into separate functions for SDK, CLI, and API
- Add new function to generate API reference sections from OpenAPI spec
- Introduce mapEndpointsById function to create a map of API endpoints
- Update run function to execute all reference generation tasks in parallel
- Import necessary new dependencies and types
* refactor(docs): migrate management api reference to app router
* fix(docs): statically build management api reference page
* fix(docs): minor styling issues
Migrates client SDK References to App Router. (Management and CLI API references aren't migrated yet, nor are self-hosting config references.)
Some notes:
Big changes to the way crawler pages are built and individual section URLs (e.g., javascript/select) are served. All of these used to be SSG-generated pages, but the number of heavy pages was just too much to handle -- slow as molasses and my laptop sounded like it was taking off, and CI sometimes refuses to build it all at all.
Tried various tricks with caching and pre-generating data but no dice.
So I changed to only building one copy of each SDK+version page, then serving the sub-URLs through a response rewrite. That's for the actual user-visible pages.
For the bot pages, each sub-URL needs to be its own page, but prebuilding it doesn't work, and rendering on demand from React components is too slow (looking for super-fast response here for SEO). Instead I changed to using an API route that serves very minimal, hand-crafted HTML. It looks ugly, but it's purely for the search bots.
You can test what bots see by running curl --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" <URL_OF_PAGE>
Also added some smoke tests to run against prod for the crawler routes, since we don't keep an eye on those regularly, and Vercel config changes could surprise-break them. Tested the meta images on Open Graph and all seems to work fine.
With this approach, full production builds are really fast: ~5 minutes
Starts using the new type spec handling, which is better at finding params automatically, so I could remove some of the manually written ones from the spec files.
