Files
supabase/apps/studio/lib
Charis 8baaa517d0 test(studio): add list_notebooks eval cases (FE-4086) (#49010)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature (test coverage) — first PR of the notebooks-evals plan, covering
FE-4086 (Evals: Assistant can list notebooks).

## What is the current behavior?

`dataset.ts` has zero notebook eval cases. Separately, two small gaps
block writing them: `assistant.eval.ts` never sets `isExplorerEnabled`,
so `NOTEBOOKS_PROMPT` never loads into the eval task's system prompt;
and `list_notebooks`' `inputSchema` has no sort parameter, even though
`getContent` already supports one, so "most recent" isn't answerable.

## What is the new behavior?

- `assistant.eval.ts` passes `isExplorerEnabled: true` so
`NOTEBOOKS_PROMPT` loads during evals.
- `list_notebooks` gains a `sort_by: 'name' | 'inserted_at'` param,
forwarded to `getContent`'s `sort`. Only creation order is exposed,
since the underlying API has no `updated_at` sort key.
- 3 new dataset cases: basic notebook enumeration, sorting by creation
time (`sort_by`/`limit` args), and a nonexistent-notebook case guarding
against hallucinated results.
- A unit test covering `sort_by` forwarding to the content API's query
param.

## Additional context

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added notebook sorting options when listing notebooks, including by
name or insertion date.
* Added evaluation coverage for listing notebooks, finding the newest
notebook, and avoiding fabricated results for nonexistent notebooks.

* **Bug Fixes**
* Ensured selected notebook sorting preferences are correctly applied
when retrieving content.
  * Improved assistant evaluation coverage with Explorer mode enabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-12 16:50:08 -04:00
..
2026-02-11 09:50:11 +01:00
2026-02-11 09:50:11 +01:00