Refactor/app router refs (#28095)

Migrates client SDK References to App Router. (Management and CLI API references aren't migrated yet, nor are self-hosting config references.)

Some notes:

Big changes to the way crawler pages are built and individual section URLs (e.g., javascript/select) are served. All of these used to be SSG-generated pages, but the number of heavy pages was just too much to handle -- slow as molasses and my laptop sounded like it was taking off, and CI sometimes refuses to build it all at all.

Tried various tricks with caching and pre-generating data but no dice.

So I changed to only building one copy of each SDK+version page, then serving the sub-URLs through a response rewrite. That's for the actual user-visible pages.

For the bot pages, each sub-URL needs to be its own page, but prebuilding it doesn't work, and rendering on demand from React components is too slow (looking for super-fast response here for SEO). Instead I changed to using an API route that serves very minimal, hand-crafted HTML. It looks ugly, but it's purely for the search bots.

You can test what bots see by running curl --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" <URL_OF_PAGE>

Also added some smoke tests to run against prod for the crawler routes, since we don't keep an eye on those regularly, and Vercel config changes could surprise-break them. Tested the meta images on Open Graph and all seems to work fine.

With this approach, full production builds are really fast: ~5 minutes

Starts using the new type spec handling, which is better at finding params automatically, so I could remove some of the manually written ones from the spec files.

.github/workflows/docs-tests-smoke.yml

@@ -0,0 +1,35 @@
+# Smoke tests against the docs production site, run daily
+
+name: Docs Production Smoke Tests
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: '0 4 * * *'
+
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          sparse-checkout: |
+            apps/docs
+            packages
+
+      - name: Use Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: '.nvmrc'
+          cache: 'npm'
+
+      - name: Install deps
+        run: npm ci
+
+      - name: Run tests
+        run: npm --prefix="apps/docs" run test:smoke