docs: enhance AGENTS.md with architecture internals and create CLAUDE.md (#1210)
Integrated into release/v3.6.5
This commit is contained in:
@@ -64,16 +64,11 @@ npm run test:protocols:e2e
|
||||
# Ecosystem compatibility tests
|
||||
npm run test:ecosystem
|
||||
|
||||
# Coverage (60% minimum for statements, lines, functions, and branches)
|
||||
# Coverage (see CONTRIBUTING.md)
|
||||
npm run test:coverage
|
||||
```
|
||||
|
||||
### PR Coverage Policy
|
||||
|
||||
- `npm run test:coverage` is the PR coverage gate in CI.
|
||||
- The repository minimum is **60%** for statements, lines, functions, and branches.
|
||||
- If a PR changes production code in `src/`, `open-sse/`, `electron/`, or `bin/`, it must include or update automated tests in the same PR.
|
||||
- For agent-driven review or coding flows: if coverage is below the gate or source changes ship without tests, do not stop at reporting. Add or update tests first, rerun the gate, and only then ask for confirmation.
|
||||
**For authoritative coverage requirements, test execution, and PR gates, see [`CONTRIBUTING.md`](CONTRIBUTING.md#running-tests).**
|
||||
|
||||
---
|
||||
|
||||
@@ -141,9 +136,69 @@ All persistence uses SQLite through domain-specific modules:
|
||||
Schema migrations live in `db/migrations/` and run via `migrationRunner.ts`.
|
||||
`src/lib/localDb.ts` is a **re-export layer only** — never add logic there.
|
||||
|
||||
#### DB Internals
|
||||
|
||||
- **`core.ts`**: `getDbInstance()` returns a singleton `better-sqlite3` instance with WAL
|
||||
journaling. `SCHEMA_SQL` defines 15 base tables. Helpers: `rowToCamel`, `encryptConnectionFields`.
|
||||
- **`migrationRunner.ts`**: Applies versioned SQL files from `db/migrations/` inside transactions.
|
||||
Tracks applied migrations in `_omniroute_migrations` table.
|
||||
- **Migrations**: 21 files (`001_initial_schema.sql` → `021_combo_call_log_targets.sql`).
|
||||
Each migration is idempotent and runs in a transaction.
|
||||
- **Domain modules** import `getDbInstance()` from `core.ts` for all CRUD operations.
|
||||
Each module owns a specific table/set of tables (e.g., `providers.ts` → `provider_connections`,
|
||||
`combos.ts` → `combos`). Encryption helpers protect sensitive fields at rest.
|
||||
- **`localDb.ts`** re-exports all domain modules — consumers import from here for convenience.
|
||||
|
||||
### API Route Layer (`src/app/api/v1/`)
|
||||
|
||||
Next.js App Router routes — each follows a consistent pattern:
|
||||
|
||||
```
|
||||
Route → CORS preflight → Body validation (Zod) → Optional auth (extractApiKey/isValidApiKey)
|
||||
→ API key policy enforcement (enforceApiKeyPolicy) → Handler delegation (open-sse)
|
||||
```
|
||||
|
||||
| Route | Handler | Notes |
|
||||
| ------------------------------- | ------------------------- | ----------------------------------------- |
|
||||
| `chat/completions/route.ts` | `handleChat()` | + prompt injection guard (clones request) |
|
||||
| `responses/route.ts` | `handleChat()` (unified) | Responses API format |
|
||||
| `embeddings/route.ts` | `handleEmbedding()` | Model listing + creation |
|
||||
| `images/generations/route.ts` | `handleImageGeneration()` | Model listing + creation |
|
||||
| `audio/transcriptions/route.ts` | audio handler | Multipart form data |
|
||||
| `audio/speech/route.ts` | TTS handler | Binary audio response |
|
||||
| `videos/generations/route.ts` | video handler | ComfyUI/SD WebUI |
|
||||
| `music/generations/route.ts` | music handler | ComfyUI workflows |
|
||||
| `moderations/route.ts` | moderation handler | Content safety |
|
||||
| `rerank/route.ts` | rerank handler | Document relevance |
|
||||
| `search/route.ts` | search handler | Web search (5 providers) |
|
||||
|
||||
**No global Next.js middleware file** — interception is route-specific. Auth is optional
|
||||
(controlled by `REQUIRE_API_KEY` env). Prompt injection guard is unique to chat completions.
|
||||
|
||||
### Request Pipeline (`open-sse/`)
|
||||
|
||||
`chatCore.ts` → executor → upstream provider. Translations in `open-sse/translator/`.
|
||||
The `open-sse/` workspace is the core streaming engine. Full request flow:
|
||||
|
||||
```
|
||||
Client Request
|
||||
→ src/app/api/v1/.../route.ts (Next.js route)
|
||||
→ open-sse/handlers/chatCore.ts::handleChatCore()
|
||||
→ Semantic/signature cache check
|
||||
→ Rate limit check (rateLimitManager)
|
||||
→ Combo routing? → open-sse/services/combo.ts::handleComboChat()
|
||||
→ resolveComboTargets() → ordered ResolvedComboTarget[]
|
||||
→ For each target: handleSingleModel() (wraps chatCore)
|
||||
→ translateRequest() (open-sse/translator/)
|
||||
→ Convert source format (e.g., OpenAI) → target format (e.g., Claude)
|
||||
→ getExecutor() → provider-specific executor instance
|
||||
→ executor.execute() (BaseExecutor → DefaultExecutor or provider-specific)
|
||||
→ buildUrl() + buildHeaders() + transformRequest()
|
||||
→ fetch() to upstream provider
|
||||
→ Retry logic with exponential backoff
|
||||
→ Response translation back to client format
|
||||
→ If Responses API: responsesTransformer.ts TransformStream
|
||||
→ SSE stream or JSON response to client
|
||||
```
|
||||
|
||||
**Handlers** (`open-sse/handlers/`): `chatCore.ts`, `responsesHandler.ts`, `embeddings.ts`,
|
||||
`imageGeneration.ts`, `videoGeneration.ts`, `musicGeneration.ts`, `audioSpeech.ts`,
|
||||
@@ -180,15 +235,46 @@ Provider-specific request executors: `base.ts`, `default.ts`, `cursor.ts`, `code
|
||||
`antigravity.ts`, `github.ts`, `gemini-cli.ts`, `kiro.ts`, `qoder.ts`, `vertex.ts`,
|
||||
`cloudflare-ai.ts`, `opencode.ts`, `pollinations.ts`, `puter.ts`.
|
||||
|
||||
#### Executor Internals
|
||||
|
||||
- **`base.ts`** (`BaseExecutor`): Abstract base with `buildUrl()`, `buildHeaders()`,
|
||||
`transformRequest()`, retry logic (exponential backoff), and `execute()`. Subclasses
|
||||
override URL/header/transform methods for provider-specific behavior.
|
||||
- **`default.ts`** (`DefaultExecutor extends BaseExecutor`): Handles most OpenAI-compatible
|
||||
providers. Reads provider config from `providerRegistry.ts` to resolve base URL, auth
|
||||
header format, and request transformations.
|
||||
- **`getExecutor()`** (`executors/index.ts`): Factory that returns the correct executor
|
||||
instance based on provider ID. Provider-specific executors (Cursor, Codex, Vertex, etc.)
|
||||
override only what differs from the default.
|
||||
|
||||
### Translator (`open-sse/translator/`)
|
||||
|
||||
Translates between API formats (OpenAI-format ↔ Anthropic, Gemini, etc.).
|
||||
Includes request/response translators with helpers for image handling.
|
||||
|
||||
#### Translator Internals
|
||||
|
||||
- **`translator/index.ts`**: Exports `translateRequest()` and format constants. Called by
|
||||
`chatCore.ts` before executor dispatch.
|
||||
- **Flow**: `translateRequest(body, sourceFormat, targetFormat)` → detects source format
|
||||
(OpenAI, Anthropic, Gemini) → applies the matching translator module → returns
|
||||
transformed body ready for the target provider.
|
||||
- **Response translation** runs in reverse after upstream response, converting back to
|
||||
the client's expected format.
|
||||
|
||||
### Transformer (`open-sse/transformer/`)
|
||||
|
||||
`responsesTransformer.ts` — transforms Responses API format to/from Chat Completions format.
|
||||
|
||||
#### Transformer Internals
|
||||
|
||||
- **`createResponsesApiTransformStream()`**: Returns a `TransformStream` that converts
|
||||
Chat Completions SSE chunks (`data: {"choices":[...]}`) into Responses API SSE events
|
||||
(`response.output_item.added`, `response.output_text.delta`, etc.).
|
||||
- Used when the client sends a Responses API request: the request is internally converted
|
||||
to Chat Completions format, dispatched normally, and the response is piped through this
|
||||
transform stream before reaching the client.
|
||||
|
||||
### Services (`open-sse/services/`)
|
||||
|
||||
36+ service modules including: `combo.ts` (routing engine), `usage.ts`, `tokenRefresh.ts`,
|
||||
@@ -198,6 +284,17 @@ Includes request/response translators with helpers for image handling.
|
||||
`emergencyFallback.ts`, `workflowFSM.ts`, `backgroundTaskDetector.ts`, `ipFilter.ts`,
|
||||
`signatureCache.ts`, `volumeDetector.ts`, `contextHandoff.ts`, and more.
|
||||
|
||||
#### Combo Routing Engine (`combo.ts`)
|
||||
|
||||
- **`handleComboChat()`**: Entry point for combo-routed requests. Receives the combo config
|
||||
and iterates through targets in order until one succeeds or all fail.
|
||||
- **`resolveComboTargets()`**: Expands a combo configuration into an ordered array of
|
||||
`ResolvedComboTarget[]`, each specifying provider + model + account + credentials.
|
||||
- **Strategies** (13): priority, weighted, fill-first, round-robin, P2C, random, least-used,
|
||||
cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay.
|
||||
- Each target calls **`handleSingleModel()`** which wraps `handleChatCore()` with
|
||||
per-target error handling and circuit breaker checks.
|
||||
|
||||
### Domain Layer (`src/domain/`)
|
||||
|
||||
Policy engine modules: `policyEngine.ts`, `comboResolver.ts`, `costRules.ts`,
|
||||
@@ -217,12 +314,37 @@ best_combo_for_task, explain_route, get_session_snapshot, sync_pricing.
|
||||
|
||||
**Skill tools** (4): skills_list, skills_enable, skills_execute, skills_executions.
|
||||
|
||||
#### MCP Internals
|
||||
|
||||
- **Tool registration**: Each tool is an object with `{ name, description, inputSchema: ZodSchema,
|
||||
handler: async (args) => {...} }`. Zod validates inputs before the handler fires.
|
||||
- **`createMcpServer()`** and **`startMcpStdio()`** exported from `mcp-server/index.ts`.
|
||||
`createMcpServer()` wires all tool sets; `startMcpStdio()` launches the stdio transport.
|
||||
- **Transports**: stdio (CLI `omniroute --mcp`), SSE (`/api/mcp/sse`), Streamable HTTP
|
||||
(`/api/mcp/stream`). All share the same tool/scope engine.
|
||||
- **Scopes** (10): Control which tool categories an API key can access. Enforcement happens
|
||||
before handler dispatch.
|
||||
- **Audit**: Every tool invocation is logged to SQLite (`mcp_audit` table) with tool name,
|
||||
args, success/failure, API key attribution, and timestamp.
|
||||
|
||||
### A2A Server (`src/lib/a2a/`)
|
||||
|
||||
JSON-RPC 2.0, SSE streaming, Task Manager with TTL cleanup(
|
||||
JSON-RPC 2.0, SSE streaming, Task Manager with TTL cleanup.
|
||||
Agent Card at `/.well-known/agent.json`.
|
||||
Skills: `quotaManagement.ts`, `smartRouting.ts`.
|
||||
|
||||
#### A2A Internals
|
||||
|
||||
- **`taskManager.ts`**: State machine lifecycle for tasks: `submitted → working →
|
||||
completed | failed | canceled`. Tasks have TTL and are cleaned up automatically.
|
||||
- **JSON-RPC methods**: `message/send` (sync), `message/stream` (SSE), `tasks/get`,
|
||||
`tasks/cancel`. Dispatched via `POST /a2a`.
|
||||
- **Skills**: Registered in a DB-backed registry. Each skill receives task context
|
||||
(messages, metadata) and returns structured results. `quotaManagement.ts` summarizes
|
||||
quota; `smartRouting.ts` recommends routing decisions.
|
||||
- **Agent Card**: `/.well-known/agent.json` exposes capabilities, skills, and metadata
|
||||
for client auto-discovery.
|
||||
|
||||
### ACP Module (`src/lib/acp/`)
|
||||
|
||||
Agent Communication Protocol registry and manager.
|
||||
@@ -237,6 +359,19 @@ conversational memory across sessions.
|
||||
Extensible skill framework: registry, executor, sandbox, built-in skills,
|
||||
custom skill support, interception, and injection.
|
||||
|
||||
#### Skills Internals
|
||||
|
||||
- **`registry.ts`**: DB-backed skill registration and discovery. Skills have metadata
|
||||
(name, description, version, enabled status) stored in SQLite.
|
||||
- **`executor.ts`**: Execution engine with configurable timeout and retry logic.
|
||||
Receives skill name + input, looks up the skill, runs it in the sandbox.
|
||||
- **`sandbox.ts`**: Isolation layer for custom (user-provided) skills. Limits resource
|
||||
access and execution time.
|
||||
- **Built-in skills**: Ship with OmniRoute (e.g., quota management, routing). Located
|
||||
alongside the registry.
|
||||
- **Interception/Injection**: Skills can intercept requests in the pipeline (pre/post
|
||||
processing) or inject context into prompts.
|
||||
|
||||
### Compliance (`src/lib/compliance/`)
|
||||
|
||||
Policy index for compliance enforcement.
|
||||
@@ -259,6 +394,14 @@ Request middleware including `promptInjectionGuard.ts`.
|
||||
|
||||
---
|
||||
|
||||
## Subdirectory AGENTS.md Files
|
||||
|
||||
- **[`open-sse/AGENTS.md`](open-sse/AGENTS.md)** — Streaming engine, request pipeline, handlers, and executors
|
||||
- **[`src/lib/db/AGENTS.md`](src/lib/db/AGENTS.md)** — SQLite persistence, domain modules, migrations
|
||||
- **[`open-sse/services/AGENTS.md`](open-sse/services/AGENTS.md)** — Routing engine, combo resolution, strategy selection
|
||||
|
||||
---
|
||||
|
||||
## Review Focus
|
||||
|
||||
- **DB ops** go through `src/lib/db/` modules, never raw SQL in routes
|
||||
|
||||
@@ -0,0 +1,225 @@
|
||||
# CLAUDE.md — AI Agent Session Bootstrap
|
||||
|
||||
> Quick-start context for AI coding agents. For deep architecture details, see `AGENTS.md`.
|
||||
> For contribution workflow, see `CONTRIBUTING.md`.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
npm install # Install deps (auto-generates .env from .env.example)
|
||||
npm run dev # Dev server at http://localhost:20128
|
||||
npm run build # Production build (Next.js 16 standalone)
|
||||
npm run lint # ESLint (0 errors expected; warnings are pre-existing)
|
||||
npm run typecheck:core # TypeScript check (should be clean)
|
||||
npm run test:coverage # Unit tests + coverage gate (60% min)
|
||||
npm run check # lint + test combined
|
||||
```
|
||||
|
||||
### Running a Single Test
|
||||
|
||||
```bash
|
||||
# Node.js native test runner (most tests)
|
||||
node --import tsx/esm --test tests/unit/your-file.test.mjs
|
||||
|
||||
# Vitest (MCP server, autoCombo, cache)
|
||||
npm run test:vitest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Project at a Glance
|
||||
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 100+ LLM providers, auto-fallback.
|
||||
|
||||
| Layer | Location | Purpose |
|
||||
| --------------- | ------------------------ | ------------------------------------------ |
|
||||
| API Routes | `src/app/api/v1/` | Next.js App Router — entry points |
|
||||
| Handlers | `open-sse/handlers/` | Request processing (chat, embeddings, etc) |
|
||||
| Executors | `open-sse/executors/` | Provider-specific HTTP dispatch |
|
||||
| Translators | `open-sse/translator/` | Format conversion (OpenAI↔Claude↔Gemini) |
|
||||
| Services | `open-sse/services/` | Combo routing, rate limits, caching, etc |
|
||||
| Database | `src/lib/db/` | SQLite domain modules (22 files) |
|
||||
| Domain/Policy | `src/domain/` | Policy engine, cost rules, fallback logic |
|
||||
| MCP Server | `open-sse/mcp-server/` | 25 tools, 3 transports, 10 scopes |
|
||||
| A2A Server | `src/lib/a2a/` | JSON-RPC 2.0 agent protocol |
|
||||
| Skills | `src/lib/skills/` | Extensible skill framework |
|
||||
| Memory | `src/lib/memory/` | Persistent conversational memory |
|
||||
| UI Components | `src/shared/components/` | React components (Tailwind CSS v4) |
|
||||
| Provider Consts | `src/shared/constants/` | Provider registry (Zod-validated) |
|
||||
| Validation | `src/shared/validation/` | Zod v4 schemas |
|
||||
| Tests | `tests/` | Unit, integration, e2e, security, load |
|
||||
|
||||
### Monorepo Layout
|
||||
|
||||
```
|
||||
OmniRoute/ # Root package
|
||||
├── src/ # Next.js 16 app (TypeScript)
|
||||
├── open-sse/ # @omniroute/open-sse workspace (streaming engine)
|
||||
├── electron/ # Desktop app (Electron)
|
||||
├── tests/ # All test suites
|
||||
├── docs/ # Documentation
|
||||
└── bin/ # CLI entry point
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Request Pipeline (Abbreviated)
|
||||
|
||||
```
|
||||
Client → /v1/chat/completions (Next.js route)
|
||||
→ CORS → Zod validation → auth? → policy check → prompt injection guard
|
||||
→ handleChatCore() [open-sse/handlers/chatCore.ts]
|
||||
→ cache check → rate limit → combo routing?
|
||||
→ resolveComboTargets() → handleSingleModel() per target
|
||||
→ translateRequest() → getExecutor() → executor.execute()
|
||||
→ fetch() upstream → retry w/ backoff
|
||||
→ response translation → SSE stream or JSON
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Conventions
|
||||
|
||||
### Code Style
|
||||
|
||||
- **2 spaces**, semicolons, double quotes, 100 char width, es5 trailing commas
|
||||
- **Imports**: external → internal (`@/`, `@omniroute/open-sse`) → relative
|
||||
- **Naming**: files=camelCase/kebab, components=PascalCase, constants=UPPER_SNAKE
|
||||
|
||||
### Database Access
|
||||
|
||||
- **Always** go through `src/lib/db/` domain modules
|
||||
- **Never** write raw SQL in routes or handlers
|
||||
- **Never** add logic to `src/lib/localDb.ts` (re-export layer only)
|
||||
- **Never** barrel-import from `localDb.ts` — import specific `db/` modules
|
||||
- DB singleton: `getDbInstance()` from `src/lib/db/core.ts` (WAL journaling)
|
||||
- Migrations: `src/lib/db/migrations/` — 21 versioned SQL files
|
||||
|
||||
### Error Handling
|
||||
|
||||
- try/catch with specific error types, log with pino context
|
||||
- Never swallow errors in SSE streams — use abort signals
|
||||
- Return proper HTTP status codes (4xx/5xx)
|
||||
|
||||
### Security
|
||||
|
||||
- **Never** commit secrets/credentials
|
||||
- **Never** use `eval()`, `new Function()`, or implied eval
|
||||
- Validate all inputs with Zod schemas
|
||||
- Encrypt credentials at rest (AES-256-GCM)
|
||||
|
||||
---
|
||||
|
||||
## Common Modification Scenarios
|
||||
|
||||
### Adding a New Provider
|
||||
|
||||
1. Register in `src/shared/constants/providers.ts` (Zod-validated at load)
|
||||
2. Add executor in `open-sse/executors/` if custom logic needed
|
||||
3. Add translator in `open-sse/translator/` if non-OpenAI format
|
||||
4. Add OAuth config in `src/lib/oauth/constants/oauth.ts` if OAuth-based
|
||||
5. Register models in `open-sse/config/providerRegistry.ts`
|
||||
6. Write tests in `tests/unit/` (registration, translation, error handling)
|
||||
|
||||
### Adding a New API Route
|
||||
|
||||
1. Create directory under `src/app/api/v1/your-route/`
|
||||
2. Create `route.ts` with `GET`/`POST` handlers
|
||||
3. Follow pattern: CORS → Zod body validation → optional auth → handler delegation
|
||||
4. Handler goes in `open-sse/handlers/` (import from there, not inline)
|
||||
5. Add tests
|
||||
|
||||
### Adding a New DB Module
|
||||
|
||||
1. Create `src/lib/db/yourModule.ts`
|
||||
2. Import `getDbInstance` from `./core.ts`
|
||||
3. Export CRUD functions for your domain table(s)
|
||||
4. Add migration in `src/lib/db/migrations/` if new tables needed
|
||||
5. Re-export from `src/lib/localDb.ts` (add to the re-export list only)
|
||||
6. Write tests
|
||||
|
||||
### Adding a New MCP Tool
|
||||
|
||||
1. Add tool definition in `open-sse/mcp-server/tools/`
|
||||
2. Define Zod input schema + async handler
|
||||
3. Register in tool set (wired by `createMcpServer()`)
|
||||
4. Assign to appropriate scope(s)
|
||||
5. Write tests (tool invocation logged to `mcp_audit` table)
|
||||
|
||||
### Adding a New A2A Skill
|
||||
|
||||
1. Create skill in `src/lib/a2a/skills/`
|
||||
2. Skill receives task context (messages, metadata) → returns structured result
|
||||
3. Register in the DB-backed skill registry
|
||||
4. Write tests
|
||||
|
||||
---
|
||||
|
||||
## Testing Cheat Sheet
|
||||
|
||||
| What | Command |
|
||||
| ----------------------- | ------------------------------------------------------- |
|
||||
| All tests | `npm run test:all` |
|
||||
| Unit tests | `npm run test:unit` |
|
||||
| Single file | `node --import tsx/esm --test tests/unit/file.test.mjs` |
|
||||
| Vitest (MCP, autoCombo) | `npm run test:vitest` |
|
||||
| E2E (Playwright) | `npm run test:e2e` |
|
||||
| Protocol E2E (MCP+A2A) | `npm run test:protocols:e2e` |
|
||||
| Ecosystem | `npm run test:ecosystem` |
|
||||
| Coverage gate | `npm run test:coverage` (60% min all metrics) |
|
||||
| Coverage report | `npm run coverage:report` |
|
||||
|
||||
**PR rule**: If you change production code in `src/`, `open-sse/`, `electron/`, or `bin/`,
|
||||
you must include or update tests in the same PR.
|
||||
|
||||
---
|
||||
|
||||
## Git Workflow
|
||||
|
||||
```bash
|
||||
# Never commit directly to main
|
||||
git checkout -b feat/your-feature
|
||||
# ... make changes ...
|
||||
git commit -m "feat: describe your change"
|
||||
git push -u origin feat/your-feature
|
||||
```
|
||||
|
||||
**Branch prefixes**: `feat/`, `fix/`, `refactor/`, `docs/`, `test/`, `chore/`
|
||||
|
||||
**Commit format** ([Conventional Commits](https://www.conventionalcommits.org/)):
|
||||
|
||||
```
|
||||
feat: add circuit breaker for provider calls
|
||||
fix: resolve JWT secret validation edge case
|
||||
docs: update AGENTS.md with pipeline internals
|
||||
test: add MCP tool unit tests
|
||||
refactor(db): consolidate rate limit tables
|
||||
```
|
||||
|
||||
**Scopes**: `db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`, `a2a`,
|
||||
`memory`, `skills`.
|
||||
|
||||
---
|
||||
|
||||
## Environment
|
||||
|
||||
- **Runtime**: Node.js ≥18 <24, ES Modules
|
||||
- **TypeScript**: 5.9, target ES2022, module esnext, resolution bundler
|
||||
- **Path aliases**: `@/*` → `src/`, `@omniroute/open-sse` → `open-sse/`
|
||||
- **Default port**: 20128 (API + dashboard on same port)
|
||||
- **Data directory**: `DATA_DIR` env var, defaults to `~/.omniroute/`
|
||||
- **Key env vars**: `PORT`, `JWT_SECRET`, `INITIAL_PASSWORD`, `REQUIRE_API_KEY`, `APP_LOG_LEVEL`
|
||||
|
||||
---
|
||||
|
||||
## Hard Rules (Never Violate)
|
||||
|
||||
1. Never commit secrets or credentials
|
||||
2. Never add logic to `localDb.ts`
|
||||
3. Never use `eval()` / `new Function()` / implied eval
|
||||
4. Never commit directly to `main`
|
||||
5. Never write raw SQL in routes — use `src/lib/db/` modules
|
||||
6. Never silently swallow errors in SSE streams
|
||||
7. Always validate inputs with Zod schemas
|
||||
8. Always include tests when changing production code
|
||||
9. Coverage must stay ≥60% (statements, lines, functions, branches)
|
||||
@@ -0,0 +1,134 @@
|
||||
# open-sse/services/ — Routing Engine & Cross-Cutting Services
|
||||
|
||||
**Purpose**: 36+ service modules powering request routing, rate limiting, quota management, token refresh, fallback strategies, and runtime state. The combo routing engine (`combo.ts`) is the core; supporting services handle resilience, accounting, and decision-making.
|
||||
|
||||
---
|
||||
|
||||
## Key Services
|
||||
|
||||
### Combo Routing Engine
|
||||
|
||||
- **`combo.ts`** (800 LOC) — Entry point for multi-model routing. **`handleComboChat()`** iterates through targets in order until success or all fail. **`resolveComboTargets()`** expands combo config into ordered `ResolvedComboTarget[]` (provider + model + account + credentials). Enforces per-target circuit breaker and fallback logic.
|
||||
- **Strategies** (13 total): `priority` (ordered list), `weighted` (probabilistic), `fill-first` (fill quota first), `round-robin`, `P2C` (power of two choices), `random`, `least-used`, `cost-optimized`, `strict-random`, `auto`, `lkgp` (last known good provider), `context-optimized`, `context-relay`.
|
||||
- **Circuit Breaker**: Per target, tracks consecutive failures; breaks after threshold, reopens on success or timeout.
|
||||
|
||||
### Quota & Rate Limiting
|
||||
|
||||
- **`rateLimitManager.ts`** — Enforces upstream rate limits (429, retry-after headers). Implements token bucket per API key + provider combo. Rejects requests exceeding limits before dispatch.
|
||||
- **`usage.ts`** — Tracks per-request token/cost consumption. Syncs with `quotaSnapshots` table. Reports cumulative usage for analytics.
|
||||
- **`quotaCache.ts`** — In-memory quota snapshots. Invalidated on write; pre-loaded at startup. Prevents DB thrashing on high-volume requests.
|
||||
|
||||
### Account & Token Management
|
||||
|
||||
- **`tokenRefresh.ts`** — Handles OAuth token expiration. Detects 401 responses, triggers refresh via provider OAuth endpoint, retries request with new token.
|
||||
- **`accountFallback.ts`** — If account reaches quota/rate-limit, switches to alternate account (combo targets). Logs account switch event.
|
||||
- **`sessionManager.ts`** — Manages request session state across retries. Tracks session ID, attempt count, fallback history.
|
||||
|
||||
### Request Routing & Intelligence
|
||||
|
||||
- **`wildcardRouter.ts`** — Matches wildcard routes in combo configs (e.g., `gpt-*` → all GPT models).
|
||||
- **`intentClassifier.ts`** — Classifies request intent (chat, embedding, image, video, etc.) for intelligent routing.
|
||||
- **`taskAwareRouter.ts`** — Routes based on task characteristics (reasoning-heavy → o1, code-gen → Cursor, long-context → Claude).
|
||||
- **`thinkingBudget.ts`** — Allocates thinking tokens for o1/o3 models; enforces per-request budget.
|
||||
- **`contextManager.ts`** — Injects routing context (system prompts, memory) into requests.
|
||||
|
||||
### Model Lifecycle & Fallback
|
||||
|
||||
- **`modelDeprecation.ts`** — Detects deprecated models (gpt-3.5, claude-2, etc.). Routes to successor models automatically.
|
||||
- **`modelFamilyFallback.ts`** — T5 intra-family fallback: if `gpt-4-turbo` unavailable, tries `gpt-4-1106-preview`, then `gpt-4`.
|
||||
- **`emergencyFallback.ts`** — Last-resort fallback when all combo targets fail. Routes to stable free provider (Qwen Code, Gemini CLI fallback).
|
||||
|
||||
### State & Detection
|
||||
|
||||
- **`workflowFSM.ts`** — Finite state machine for multi-turn workflows (prompt engineering → execution → validation).
|
||||
- **`backgroundTaskDetector.ts`** — Detects long-running background tasks; routes to batch APIs or defers execution.
|
||||
- **`ipFilter.ts`** — IP-based routing rules (geographic or access control).
|
||||
- **`signatureCache.ts`** — Caches request signatures for duplicate detection and deduplication.
|
||||
- **`volumeDetector.ts`** — Detects request volume spikes; triggers rate-limit escalation or load-shedding.
|
||||
- **`contextHandoff.ts`** — Serializes/restores session context for agent handoff (A2A protocol).
|
||||
|
||||
### Auto-Routing & Adaptive
|
||||
|
||||
- **`autoCombo/`** — Auto-generates combo configs based on historical performance, cost, and latency.
|
||||
- **`modelFamilyFallback.ts`** — Automatic fallback within model families (T5, GPT-4, Claude).
|
||||
|
||||
### Advanced Services
|
||||
|
||||
- **`promptInjectionGuard.ts`** (middleware) — Clones request, sanitizes user input, detects prompt injection patterns before dispatch
|
||||
- **`costRules.ts`** (domain layer) — Cost-based routing decisions (cheapest-first, within budget)
|
||||
- **`degradation.ts`** (domain layer) — Handles service degradation scenarios (provider down, quota exceeded)
|
||||
- **`resilience.ts`** — Retry logic, exponential backoff, circuit breaker orchestration across all services
|
||||
|
||||
---
|
||||
|
||||
## Complexity Hotspots
|
||||
|
||||
| Module | Lines | Risk | Mitigation |
|
||||
| ------------------------ | ----- | ---------------------------------------------------------- | ------------------------------------------------------------------------------ |
|
||||
| `combo.ts` | ~800 | High — routing logic, strategy dispatch, fallback ordering | Unit tests for each strategy, integration tests for combo sequences |
|
||||
| `providerRegistry.ts` | 3000+ | High — 100+ provider configs, executor dispatch | Auto-validate via Zod at module load, split into provider-specific sub-modules |
|
||||
| `rateLimitManager.ts` | ~300 | Medium — token bucket state, concurrent requests | Unit tests for bucket refill, edge cases (clock skew, parallel requests) |
|
||||
| `modelFamilyFallback.ts` | ~200 | Medium — fallback chains, family detection | Test all family chains, ensure no circular fallbacks |
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
Each service requires unit and integration tests. For authoritative coverage requirements and test execution guidelines, see [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests).
|
||||
|
||||
- **Unit tests** — Each service in isolation with mocked dependencies (combos, models, executors)
|
||||
- **Integration tests** — Combo routing with real combo configs, verify target resolution and fallback behavior
|
||||
- **E2E tests** — Full request flow: chat → combo routing → provider selection → response streaming
|
||||
- **Chaos tests** — Simulate provider failures, rate limits, token expiration; verify graceful degradation
|
||||
- **Benchmarks** — Measure routing latency, combo resolution time (target: <10ms for 50 targets)
|
||||
|
||||
---
|
||||
|
||||
## Performance Constraints
|
||||
|
||||
- **Combo resolution**: <10ms for typical configs (5–20 targets)
|
||||
- **Rate limit checks**: <1ms (in-memory token bucket)
|
||||
- **Model family fallback**: <5ms (cached family definitions)
|
||||
- **Request routing dispatch**: <2ms (hot path, pre-computed strategy dispatch)
|
||||
- **No blocking I/O** in routing hot path — all async, no awaits on DB queries outside context injection
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- ❌ Synchronous DB calls in `combo.ts` hot path — pre-compute and cache
|
||||
- ❌ Retry logic in handlers; use `retry()` from resilience service
|
||||
- ❌ Direct provider config access; use `providerRegistry` getter functions
|
||||
- ❌ Hardcoded fallback chains; define in `modelFamilyFallback.ts` instead
|
||||
- ❌ State mutations across concurrent requests; use request-scoped context only
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Service
|
||||
|
||||
1. Create `open-sse/services/[serviceName].ts` with clear responsibilities
|
||||
2. Export main handler function and any constants
|
||||
3. Add unit tests in `tests/unit/services/[serviceName].test.mjs`
|
||||
4. Integrate into request pipeline in `handlers/chatCore.ts` (if routing-related) or expose via combo.ts
|
||||
5. Update routing logic in `combo.ts` if service affects target selection or fallback
|
||||
6. Document in this file (table, key decisions section)
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Combo-first design**: All routing decisions go through combo engine; fallback strategies are combo targets, not ad-hoc logic
|
||||
- **Service composition**: Small focused modules; combo.ts orchestrates them, not monolithic routing
|
||||
- **Circuit breaker per-target**: Failures isolated to specific provider+account combo; other targets unaffected
|
||||
- **Caching everywhere**: Models, providers, quotas, family fallbacks all pre-cached; invalidated on write
|
||||
- **13 strategies** over hardcoded logic: Strategy pattern allows new routing logic without touching combo.ts core
|
||||
|
||||
---
|
||||
|
||||
## Review Focus
|
||||
|
||||
- New services must not add blocking I/O to routing hot path
|
||||
- Combo target resolution under 10ms (measure with benchmarks)
|
||||
- Circuit breaker state per-target, not global
|
||||
- All fallback chains tested (no infinite loops)
|
||||
- Coverage requirements: See [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) (60% gate enforced in CI)
|
||||
@@ -0,0 +1,102 @@
|
||||
# src/lib/db/ — SQLite Persistence Layer
|
||||
|
||||
**Purpose**: Domain-driven SQLite persistence. Each module owns a specific table set. Schema migrations are versioned and idempotent. No raw SQL in routes — all ops go through `src/lib/db/` modules.
|
||||
|
||||
---
|
||||
|
||||
## Key Modules
|
||||
|
||||
### Core Infrastructure
|
||||
|
||||
- **`core.ts`** — `getDbInstance()` returns singleton `better-sqlite3` with WAL journaling. Exports `rowToCamel()` (snake_case → camelCase), `encryptConnectionFields()` for provider credentials at rest. `SCHEMA_SQL` defines 15 base tables.
|
||||
- **`migrationRunner.ts`** — Applies versioned SQL files from `db/migrations/` inside transactions. Tracks applied migrations in `_omniroute_migrations`. Runs at startup; each migration is idempotent.
|
||||
- **`db/migrations/`** — 21 SQL files (`001_initial_schema.sql` → `021_combo_call_log_targets.sql`). Each migration has single responsibility, runs in a transaction, never fails partially.
|
||||
- **`localDb.ts`** — Re-export layer only. Never add logic here. Consumers import domain modules from this file for convenience.
|
||||
|
||||
### Domain Modules (22 total)
|
||||
|
||||
Each module owns specific tables + CRUD operations:
|
||||
|
||||
| Module | Tables | Responsibility |
|
||||
| ----------------------- | ------------------------- | ------------------------------------------------------- |
|
||||
| `providers.ts` | `provider_connections` | OAuth/API key provider registration and credentials |
|
||||
| `models.ts` | `models` | Model definitions, capabilities, pricing |
|
||||
| `combos.ts` | `combos`, `combo_targets` | Combo routing configs, target ordering |
|
||||
| `apiKeys.ts` | `api_keys` | API key lifecycle, scopes, quota tracking |
|
||||
| `settings.ts` | `settings` | KV store for system configuration |
|
||||
| `backup.ts` | Backup export/import ops | Serialize/deserialize entire DB state |
|
||||
| `proxies.ts` | `proxies` | MITM proxy configs and routing rules |
|
||||
| `prompts.ts` | `prompts` | Reusable prompt templates, versioning |
|
||||
| `webhooks.ts` | `webhooks` | Event-driven webhook subscriptions and logs |
|
||||
| `detailedLogs.ts` | `detailed_logs` | Per-request audit logging (optional, high volume) |
|
||||
| `domainState.ts` | `domain_state` | Transient runtime state (not persisted across restarts) |
|
||||
| `registeredKeys.ts` | `registered_keys` | Whitelisted API keys for MCP/A2A access |
|
||||
| `quotaSnapshots.ts` | `quota_snapshots` | Historical quota usage for analytics |
|
||||
| `modelComboMappings.ts` | `model_combo_mappings` | Map models to combo defaults |
|
||||
| `cliToolState.ts` | `cli_tool_state` | CLI-specific persistent state |
|
||||
| `encryption.ts` | — | Helpers for encrypting/decrypting sensitive fields |
|
||||
| `readCache.ts` | — | In-memory cache for read-heavy ops (models, providers) |
|
||||
| `secrets.ts` | `secrets` | Encrypted secret storage (API keys at rest) |
|
||||
| `stateReset.ts` | — | Wipe/reset DB state for testing or recovery |
|
||||
| `contextHandoffs.ts` | `context_handoffs` | Store/retrieve session context for agent handoff |
|
||||
| `migrations/` | — | Versioned SQL schema evolution |
|
||||
| `core.ts` | — | Singleton DB instance, helpers, schema definition |
|
||||
|
||||
### Encryption & Security
|
||||
|
||||
- **Sensitive fields** (API keys, OAuth tokens, connection strings) encrypted at rest using `src/lib/encryption/` utilities
|
||||
- **`encryptConnectionFields()`** in `core.ts` — Automatic encryption when storing provider credentials
|
||||
- **`secrets.ts`** — Dedicated encrypted store for long-term secret handling
|
||||
- **Never log** SQLite encryption keys or raw secrets; always use redacted values in logs
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
For authoritative coverage requirements and test execution guidelines, see [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) (lines 136–162).
|
||||
|
||||
- **Unit tests** mock `getDbInstance()` to return isolated sqlite in-memory instance
|
||||
- **Integration tests** use real SQLite with migrations applied, data cleaned up after each test
|
||||
- **No fixture interdependencies** — each test runs migrations fresh
|
||||
- Test files: `tests/unit/db/*.test.mjs`, `tests/integration/db/*.test.mjs`
|
||||
|
||||
### Anti-Patterns
|
||||
|
||||
- ❌ Raw SQL in routes — always use domain module functions
|
||||
- ❌ Direct `prepare()` statements outside `db/` modules — breaks modularity
|
||||
- ❌ Mixing encryption logic in domain modules — use `encryption.ts` helpers only
|
||||
- ❌ Accessing `provider_connections` table from `combos.ts` — each module owns its tables
|
||||
- ❌ Skipping migrations for schema changes — all changes go through `db/migrations/`
|
||||
|
||||
### Adding a New Domain Module
|
||||
|
||||
1. Create `src/lib/db/[module].ts` with CRUD functions (create, read, update, delete, list)
|
||||
2. Export from `src/lib/db/localDb.ts` (add re-export)
|
||||
3. If new tables required: create migration in `db/migrations/NNN_[description].sql`
|
||||
4. Run migration via `migrationRunner.ts` at startup (automatic)
|
||||
5. Add unit tests in `tests/unit/db/[module].test.mjs`
|
||||
6. Ensure tests meet the coverage requirements in [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests)
|
||||
|
||||
### Performance Notes
|
||||
|
||||
- **Read cache** (`readCache.ts`) — Pre-loads frequently accessed data (models, providers) at startup; invalidated on write
|
||||
- **WAL journaling** (`core.ts`) — Enables concurrent reads during writes
|
||||
- **Batch operations** — Use prepared statements with parameter binding to avoid SQL injection
|
||||
- **Connection pooling** — Singleton pattern prevents per-request connection overhead
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **SQLite over PostgreSQL**: Simpler deployment, no separate database server, encryption at application layer
|
||||
- **Versioned migrations**: Each schema change is tracked, reproducible, reversible with effort
|
||||
- **Domain modules**: Enforces single responsibility, prevents cross-module table access
|
||||
- **Re-export layer**: Convenience for consumers; `localDb.ts` is re-export-only to prevent circular dependencies
|
||||
|
||||
---
|
||||
|
||||
## Review Focus
|
||||
|
||||
- DB module changes must preserve domain boundaries (one module = one table set)
|
||||
- New migrations must be idempotent and run inside transactions
|
||||
- Encryption helpers used for all sensitive fields
|
||||
- Test coverage and PR requirements: see [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) (lines 136–162)
|
||||
- No raw SQL in routes or non-db modules
|
||||
Reference in New Issue
Block a user