Files
supabase/apps/docs/app/api/crawlers/route.smoke.test.ts
Charis fc164b5d07 Refactor/app router refs (#28095)
Migrates client SDK References to App Router. (Management and CLI API references aren't migrated yet, nor are self-hosting config references.)

Some notes:

Big changes to the way crawler pages are built and individual section URLs (e.g., javascript/select) are served. All of these used to be SSG-generated pages, but the number of heavy pages was just too much to handle -- slow as molasses and my laptop sounded like it was taking off, and CI sometimes refuses to build it all at all.

Tried various tricks with caching and pre-generating data but no dice.

So I changed to only building one copy of each SDK+version page, then serving the sub-URLs through a response rewrite. That's for the actual user-visible pages.

For the bot pages, each sub-URL needs to be its own page, but prebuilding it doesn't work, and rendering on demand from React components is too slow (looking for super-fast response here for SEO). Instead I changed to using an API route that serves very minimal, hand-crafted HTML. It looks ugly, but it's purely for the search bots.

You can test what bots see by running curl --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" <URL_OF_PAGE>

Also added some smoke tests to run against prod for the crawler routes, since we don't keep an eye on those regularly, and Vercel config changes could surprise-break them. Tested the meta images on Open Graph and all seems to work fine.

With this approach, full production builds are really fast: ~5 minutes

Starts using the new type spec handling, which is better at finding params automatically, so I could remove some of the manually written ones from the spec files.
2024-08-13 16:12:59 -04:00

73 lines
2.6 KiB
TypeScript

import { load } from 'cheerio'
import { describe, expect, it } from 'vitest'
const REFERENCE_DOCS_URL = 'https://supabase.com/docs/reference'
// For dev testing: comment out above and uncomment below
// const REFERENCE_DOCS_URL = 'http://localhost:3001/docs/reference'
describe('prod smoke test: crawler pages return correct data', () => {
/**
* No special tricks required to spoof the user agent. Tests are correctly
* detected as coming from bots. If they ever aren't, the `h1` test will fail
* as a different `h1` is served to non-bots.
*/
it('metadata: title, description, canonical, image', async () => {
const result = await fetch(REFERENCE_DOCS_URL + '/javascript/rangelte')
const text = await result.text()
const $ = load(text)
const title = $('title').text()
expect(title).toBe('JavaScript: Less than or equal to a range | Supabase Docs')
const metaDescription = $('meta[name="description"]')
expect(metaDescription.attr('content')).toBe(
'Supabase API reference for JavaScript: Less than or equal to a range'
)
const canonical = $('link[rel="canonical"]')
expect(canonical.attr('href')).toBe('https://supabase.com/docs/reference/javascript/rangelte')
const ogImage = $('meta[name="og:image"]')
expect(ogImage.attr('content')).toBe('https://supabase.com/docs/img/supabase-og-image.png')
const twitterImage = $('meta[name="twitter:image"]')
expect(twitterImage.attr('content')).toBe('https://supabase.com/docs/img/supabase-og-image.png')
})
it('markdown pages', async () => {
const result = await fetch(REFERENCE_DOCS_URL + '/javascript/introduction')
const text = await result.text()
const $ = load(text)
const h1 = $('h1').text()
expect(h1).toBe('JavaScript: Introduction')
const firstPara = $('h1').next().text()
expect(/JavaScript library/.test(firstPara)).toBe(true)
expect(/supabase-js/.test(firstPara)).toBe(true)
})
it('function pages', async () => {
const result = await fetch(REFERENCE_DOCS_URL + '/javascript/rangelte')
const text = await result.text()
const $ = load(text)
const h1 = $('h1').text()
expect(h1).toBe('JavaScript: Less than or equal to a range')
const description = $('h1').next().text()
expect(description).toBe(
'Only relevant for range columns. Match only rows where every element in column is either contained in range or less than any element in range.'
)
const headings = [] as Array<string | undefined>
$('h2').map(function () {
headings.push($(this).attr('id'))
})
expect(headings.includes('parameters')).toBe(true)
expect(headings.includes('examples')).toBe(true)
})
})