diff --git a/AGENTS.md b/AGENTS.md index 134a0b64..2b3504b1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -64,16 +64,11 @@ npm run test:protocols:e2e # Ecosystem compatibility tests npm run test:ecosystem -# Coverage (60% minimum for statements, lines, functions, and branches) +# Coverage (see CONTRIBUTING.md) npm run test:coverage ``` -### PR Coverage Policy - -- `npm run test:coverage` is the PR coverage gate in CI. -- The repository minimum is **60%** for statements, lines, functions, and branches. -- If a PR changes production code in `src/`, `open-sse/`, `electron/`, or `bin/`, it must include or update automated tests in the same PR. -- For agent-driven review or coding flows: if coverage is below the gate or source changes ship without tests, do not stop at reporting. Add or update tests first, rerun the gate, and only then ask for confirmation. +**For authoritative coverage requirements, test execution, and PR gates, see [`CONTRIBUTING.md`](CONTRIBUTING.md#running-tests).** --- @@ -141,9 +136,69 @@ All persistence uses SQLite through domain-specific modules: Schema migrations live in `db/migrations/` and run via `migrationRunner.ts`. `src/lib/localDb.ts` is a **re-export layer only** — never add logic there. +#### DB Internals + +- **`core.ts`**: `getDbInstance()` returns a singleton `better-sqlite3` instance with WAL + journaling. `SCHEMA_SQL` defines 15 base tables. Helpers: `rowToCamel`, `encryptConnectionFields`. +- **`migrationRunner.ts`**: Applies versioned SQL files from `db/migrations/` inside transactions. + Tracks applied migrations in `_omniroute_migrations` table. +- **Migrations**: 21 files (`001_initial_schema.sql` → `021_combo_call_log_targets.sql`). + Each migration is idempotent and runs in a transaction. +- **Domain modules** import `getDbInstance()` from `core.ts` for all CRUD operations. + Each module owns a specific table/set of tables (e.g., `providers.ts` → `provider_connections`, + `combos.ts` → `combos`). Encryption helpers protect sensitive fields at rest. +- **`localDb.ts`** re-exports all domain modules — consumers import from here for convenience. + +### API Route Layer (`src/app/api/v1/`) + +Next.js App Router routes — each follows a consistent pattern: + +``` +Route → CORS preflight → Body validation (Zod) → Optional auth (extractApiKey/isValidApiKey) + → API key policy enforcement (enforceApiKeyPolicy) → Handler delegation (open-sse) +``` + +| Route | Handler | Notes | +| ------------------------------- | ------------------------- | ----------------------------------------- | +| `chat/completions/route.ts` | `handleChat()` | + prompt injection guard (clones request) | +| `responses/route.ts` | `handleChat()` (unified) | Responses API format | +| `embeddings/route.ts` | `handleEmbedding()` | Model listing + creation | +| `images/generations/route.ts` | `handleImageGeneration()` | Model listing + creation | +| `audio/transcriptions/route.ts` | audio handler | Multipart form data | +| `audio/speech/route.ts` | TTS handler | Binary audio response | +| `videos/generations/route.ts` | video handler | ComfyUI/SD WebUI | +| `music/generations/route.ts` | music handler | ComfyUI workflows | +| `moderations/route.ts` | moderation handler | Content safety | +| `rerank/route.ts` | rerank handler | Document relevance | +| `search/route.ts` | search handler | Web search (5 providers) | + +**No global Next.js middleware file** — interception is route-specific. Auth is optional +(controlled by `REQUIRE_API_KEY` env). Prompt injection guard is unique to chat completions. + ### Request Pipeline (`open-sse/`) -`chatCore.ts` → executor → upstream provider. Translations in `open-sse/translator/`. +The `open-sse/` workspace is the core streaming engine. Full request flow: + +``` +Client Request + → src/app/api/v1/.../route.ts (Next.js route) + → open-sse/handlers/chatCore.ts::handleChatCore() + → Semantic/signature cache check + → Rate limit check (rateLimitManager) + → Combo routing? → open-sse/services/combo.ts::handleComboChat() + → resolveComboTargets() → ordered ResolvedComboTarget[] + → For each target: handleSingleModel() (wraps chatCore) + → translateRequest() (open-sse/translator/) + → Convert source format (e.g., OpenAI) → target format (e.g., Claude) + → getExecutor() → provider-specific executor instance + → executor.execute() (BaseExecutor → DefaultExecutor or provider-specific) + → buildUrl() + buildHeaders() + transformRequest() + → fetch() to upstream provider + → Retry logic with exponential backoff + → Response translation back to client format + → If Responses API: responsesTransformer.ts TransformStream + → SSE stream or JSON response to client +``` **Handlers** (`open-sse/handlers/`): `chatCore.ts`, `responsesHandler.ts`, `embeddings.ts`, `imageGeneration.ts`, `videoGeneration.ts`, `musicGeneration.ts`, `audioSpeech.ts`, @@ -180,15 +235,46 @@ Provider-specific request executors: `base.ts`, `default.ts`, `cursor.ts`, `code `antigravity.ts`, `github.ts`, `gemini-cli.ts`, `kiro.ts`, `qoder.ts`, `vertex.ts`, `cloudflare-ai.ts`, `opencode.ts`, `pollinations.ts`, `puter.ts`. +#### Executor Internals + +- **`base.ts`** (`BaseExecutor`): Abstract base with `buildUrl()`, `buildHeaders()`, + `transformRequest()`, retry logic (exponential backoff), and `execute()`. Subclasses + override URL/header/transform methods for provider-specific behavior. +- **`default.ts`** (`DefaultExecutor extends BaseExecutor`): Handles most OpenAI-compatible + providers. Reads provider config from `providerRegistry.ts` to resolve base URL, auth + header format, and request transformations. +- **`getExecutor()`** (`executors/index.ts`): Factory that returns the correct executor + instance based on provider ID. Provider-specific executors (Cursor, Codex, Vertex, etc.) + override only what differs from the default. + ### Translator (`open-sse/translator/`) Translates between API formats (OpenAI-format ↔ Anthropic, Gemini, etc.). Includes request/response translators with helpers for image handling. +#### Translator Internals + +- **`translator/index.ts`**: Exports `translateRequest()` and format constants. Called by + `chatCore.ts` before executor dispatch. +- **Flow**: `translateRequest(body, sourceFormat, targetFormat)` → detects source format + (OpenAI, Anthropic, Gemini) → applies the matching translator module → returns + transformed body ready for the target provider. +- **Response translation** runs in reverse after upstream response, converting back to + the client's expected format. + ### Transformer (`open-sse/transformer/`) `responsesTransformer.ts` — transforms Responses API format to/from Chat Completions format. +#### Transformer Internals + +- **`createResponsesApiTransformStream()`**: Returns a `TransformStream` that converts + Chat Completions SSE chunks (`data: {"choices":[...]}`) into Responses API SSE events + (`response.output_item.added`, `response.output_text.delta`, etc.). +- Used when the client sends a Responses API request: the request is internally converted + to Chat Completions format, dispatched normally, and the response is piped through this + transform stream before reaching the client. + ### Services (`open-sse/services/`) 36+ service modules including: `combo.ts` (routing engine), `usage.ts`, `tokenRefresh.ts`, @@ -198,6 +284,17 @@ Includes request/response translators with helpers for image handling. `emergencyFallback.ts`, `workflowFSM.ts`, `backgroundTaskDetector.ts`, `ipFilter.ts`, `signatureCache.ts`, `volumeDetector.ts`, `contextHandoff.ts`, and more. +#### Combo Routing Engine (`combo.ts`) + +- **`handleComboChat()`**: Entry point for combo-routed requests. Receives the combo config + and iterates through targets in order until one succeeds or all fail. +- **`resolveComboTargets()`**: Expands a combo configuration into an ordered array of + `ResolvedComboTarget[]`, each specifying provider + model + account + credentials. +- **Strategies** (13): priority, weighted, fill-first, round-robin, P2C, random, least-used, + cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay. +- Each target calls **`handleSingleModel()`** which wraps `handleChatCore()` with + per-target error handling and circuit breaker checks. + ### Domain Layer (`src/domain/`) Policy engine modules: `policyEngine.ts`, `comboResolver.ts`, `costRules.ts`, @@ -217,12 +314,37 @@ best_combo_for_task, explain_route, get_session_snapshot, sync_pricing. **Skill tools** (4): skills_list, skills_enable, skills_execute, skills_executions. +#### MCP Internals + +- **Tool registration**: Each tool is an object with `{ name, description, inputSchema: ZodSchema, +handler: async (args) => {...} }`. Zod validates inputs before the handler fires. +- **`createMcpServer()`** and **`startMcpStdio()`** exported from `mcp-server/index.ts`. + `createMcpServer()` wires all tool sets; `startMcpStdio()` launches the stdio transport. +- **Transports**: stdio (CLI `omniroute --mcp`), SSE (`/api/mcp/sse`), Streamable HTTP + (`/api/mcp/stream`). All share the same tool/scope engine. +- **Scopes** (10): Control which tool categories an API key can access. Enforcement happens + before handler dispatch. +- **Audit**: Every tool invocation is logged to SQLite (`mcp_audit` table) with tool name, + args, success/failure, API key attribution, and timestamp. + ### A2A Server (`src/lib/a2a/`) -JSON-RPC 2.0, SSE streaming, Task Manager with TTL cleanup( +JSON-RPC 2.0, SSE streaming, Task Manager with TTL cleanup. Agent Card at `/.well-known/agent.json`. Skills: `quotaManagement.ts`, `smartRouting.ts`. +#### A2A Internals + +- **`taskManager.ts`**: State machine lifecycle for tasks: `submitted → working → +completed | failed | canceled`. Tasks have TTL and are cleaned up automatically. +- **JSON-RPC methods**: `message/send` (sync), `message/stream` (SSE), `tasks/get`, + `tasks/cancel`. Dispatched via `POST /a2a`. +- **Skills**: Registered in a DB-backed registry. Each skill receives task context + (messages, metadata) and returns structured results. `quotaManagement.ts` summarizes + quota; `smartRouting.ts` recommends routing decisions. +- **Agent Card**: `/.well-known/agent.json` exposes capabilities, skills, and metadata + for client auto-discovery. + ### ACP Module (`src/lib/acp/`) Agent Communication Protocol registry and manager. @@ -237,6 +359,19 @@ conversational memory across sessions. Extensible skill framework: registry, executor, sandbox, built-in skills, custom skill support, interception, and injection. +#### Skills Internals + +- **`registry.ts`**: DB-backed skill registration and discovery. Skills have metadata + (name, description, version, enabled status) stored in SQLite. +- **`executor.ts`**: Execution engine with configurable timeout and retry logic. + Receives skill name + input, looks up the skill, runs it in the sandbox. +- **`sandbox.ts`**: Isolation layer for custom (user-provided) skills. Limits resource + access and execution time. +- **Built-in skills**: Ship with OmniRoute (e.g., quota management, routing). Located + alongside the registry. +- **Interception/Injection**: Skills can intercept requests in the pipeline (pre/post + processing) or inject context into prompts. + ### Compliance (`src/lib/compliance/`) Policy index for compliance enforcement. @@ -259,6 +394,14 @@ Request middleware including `promptInjectionGuard.ts`. --- +## Subdirectory AGENTS.md Files + +- **[`open-sse/AGENTS.md`](open-sse/AGENTS.md)** — Streaming engine, request pipeline, handlers, and executors +- **[`src/lib/db/AGENTS.md`](src/lib/db/AGENTS.md)** — SQLite persistence, domain modules, migrations +- **[`open-sse/services/AGENTS.md`](open-sse/services/AGENTS.md)** — Routing engine, combo resolution, strategy selection + +--- + ## Review Focus - **DB ops** go through `src/lib/db/` modules, never raw SQL in routes diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..5f806ee4 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,225 @@ +# CLAUDE.md — AI Agent Session Bootstrap + +> Quick-start context for AI coding agents. For deep architecture details, see `AGENTS.md`. +> For contribution workflow, see `CONTRIBUTING.md`. + +## Quick Start + +```bash +npm install # Install deps (auto-generates .env from .env.example) +npm run dev # Dev server at http://localhost:20128 +npm run build # Production build (Next.js 16 standalone) +npm run lint # ESLint (0 errors expected; warnings are pre-existing) +npm run typecheck:core # TypeScript check (should be clean) +npm run test:coverage # Unit tests + coverage gate (60% min) +npm run check # lint + test combined +``` + +### Running a Single Test + +```bash +# Node.js native test runner (most tests) +node --import tsx/esm --test tests/unit/your-file.test.mjs + +# Vitest (MCP server, autoCombo, cache) +npm run test:vitest +``` + +--- + +## Project at a Glance + +**OmniRoute** — unified AI proxy/router. One endpoint, 100+ LLM providers, auto-fallback. + +| Layer | Location | Purpose | +| --------------- | ------------------------ | ------------------------------------------ | +| API Routes | `src/app/api/v1/` | Next.js App Router — entry points | +| Handlers | `open-sse/handlers/` | Request processing (chat, embeddings, etc) | +| Executors | `open-sse/executors/` | Provider-specific HTTP dispatch | +| Translators | `open-sse/translator/` | Format conversion (OpenAI↔Claude↔Gemini) | +| Services | `open-sse/services/` | Combo routing, rate limits, caching, etc | +| Database | `src/lib/db/` | SQLite domain modules (22 files) | +| Domain/Policy | `src/domain/` | Policy engine, cost rules, fallback logic | +| MCP Server | `open-sse/mcp-server/` | 25 tools, 3 transports, 10 scopes | +| A2A Server | `src/lib/a2a/` | JSON-RPC 2.0 agent protocol | +| Skills | `src/lib/skills/` | Extensible skill framework | +| Memory | `src/lib/memory/` | Persistent conversational memory | +| UI Components | `src/shared/components/` | React components (Tailwind CSS v4) | +| Provider Consts | `src/shared/constants/` | Provider registry (Zod-validated) | +| Validation | `src/shared/validation/` | Zod v4 schemas | +| Tests | `tests/` | Unit, integration, e2e, security, load | + +### Monorepo Layout + +``` +OmniRoute/ # Root package +├── src/ # Next.js 16 app (TypeScript) +├── open-sse/ # @omniroute/open-sse workspace (streaming engine) +├── electron/ # Desktop app (Electron) +├── tests/ # All test suites +├── docs/ # Documentation +└── bin/ # CLI entry point +``` + +--- + +## Request Pipeline (Abbreviated) + +``` +Client → /v1/chat/completions (Next.js route) + → CORS → Zod validation → auth? → policy check → prompt injection guard + → handleChatCore() [open-sse/handlers/chatCore.ts] + → cache check → rate limit → combo routing? + → resolveComboTargets() → handleSingleModel() per target + → translateRequest() → getExecutor() → executor.execute() + → fetch() upstream → retry w/ backoff + → response translation → SSE stream or JSON +``` + +--- + +## Key Conventions + +### Code Style + +- **2 spaces**, semicolons, double quotes, 100 char width, es5 trailing commas +- **Imports**: external → internal (`@/`, `@omniroute/open-sse`) → relative +- **Naming**: files=camelCase/kebab, components=PascalCase, constants=UPPER_SNAKE + +### Database Access + +- **Always** go through `src/lib/db/` domain modules +- **Never** write raw SQL in routes or handlers +- **Never** add logic to `src/lib/localDb.ts` (re-export layer only) +- **Never** barrel-import from `localDb.ts` — import specific `db/` modules +- DB singleton: `getDbInstance()` from `src/lib/db/core.ts` (WAL journaling) +- Migrations: `src/lib/db/migrations/` — 21 versioned SQL files + +### Error Handling + +- try/catch with specific error types, log with pino context +- Never swallow errors in SSE streams — use abort signals +- Return proper HTTP status codes (4xx/5xx) + +### Security + +- **Never** commit secrets/credentials +- **Never** use `eval()`, `new Function()`, or implied eval +- Validate all inputs with Zod schemas +- Encrypt credentials at rest (AES-256-GCM) + +--- + +## Common Modification Scenarios + +### Adding a New Provider + +1. Register in `src/shared/constants/providers.ts` (Zod-validated at load) +2. Add executor in `open-sse/executors/` if custom logic needed +3. Add translator in `open-sse/translator/` if non-OpenAI format +4. Add OAuth config in `src/lib/oauth/constants/oauth.ts` if OAuth-based +5. Register models in `open-sse/config/providerRegistry.ts` +6. Write tests in `tests/unit/` (registration, translation, error handling) + +### Adding a New API Route + +1. Create directory under `src/app/api/v1/your-route/` +2. Create `route.ts` with `GET`/`POST` handlers +3. Follow pattern: CORS → Zod body validation → optional auth → handler delegation +4. Handler goes in `open-sse/handlers/` (import from there, not inline) +5. Add tests + +### Adding a New DB Module + +1. Create `src/lib/db/yourModule.ts` +2. Import `getDbInstance` from `./core.ts` +3. Export CRUD functions for your domain table(s) +4. Add migration in `src/lib/db/migrations/` if new tables needed +5. Re-export from `src/lib/localDb.ts` (add to the re-export list only) +6. Write tests + +### Adding a New MCP Tool + +1. Add tool definition in `open-sse/mcp-server/tools/` +2. Define Zod input schema + async handler +3. Register in tool set (wired by `createMcpServer()`) +4. Assign to appropriate scope(s) +5. Write tests (tool invocation logged to `mcp_audit` table) + +### Adding a New A2A Skill + +1. Create skill in `src/lib/a2a/skills/` +2. Skill receives task context (messages, metadata) → returns structured result +3. Register in the DB-backed skill registry +4. Write tests + +--- + +## Testing Cheat Sheet + +| What | Command | +| ----------------------- | ------------------------------------------------------- | +| All tests | `npm run test:all` | +| Unit tests | `npm run test:unit` | +| Single file | `node --import tsx/esm --test tests/unit/file.test.mjs` | +| Vitest (MCP, autoCombo) | `npm run test:vitest` | +| E2E (Playwright) | `npm run test:e2e` | +| Protocol E2E (MCP+A2A) | `npm run test:protocols:e2e` | +| Ecosystem | `npm run test:ecosystem` | +| Coverage gate | `npm run test:coverage` (60% min all metrics) | +| Coverage report | `npm run coverage:report` | + +**PR rule**: If you change production code in `src/`, `open-sse/`, `electron/`, or `bin/`, +you must include or update tests in the same PR. + +--- + +## Git Workflow + +```bash +# Never commit directly to main +git checkout -b feat/your-feature +# ... make changes ... +git commit -m "feat: describe your change" +git push -u origin feat/your-feature +``` + +**Branch prefixes**: `feat/`, `fix/`, `refactor/`, `docs/`, `test/`, `chore/` + +**Commit format** ([Conventional Commits](https://www.conventionalcommits.org/)): + +``` +feat: add circuit breaker for provider calls +fix: resolve JWT secret validation edge case +docs: update AGENTS.md with pipeline internals +test: add MCP tool unit tests +refactor(db): consolidate rate limit tables +``` + +**Scopes**: `db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`, `a2a`, +`memory`, `skills`. + +--- + +## Environment + +- **Runtime**: Node.js ≥18 <24, ES Modules +- **TypeScript**: 5.9, target ES2022, module esnext, resolution bundler +- **Path aliases**: `@/*` → `src/`, `@omniroute/open-sse` → `open-sse/` +- **Default port**: 20128 (API + dashboard on same port) +- **Data directory**: `DATA_DIR` env var, defaults to `~/.omniroute/` +- **Key env vars**: `PORT`, `JWT_SECRET`, `INITIAL_PASSWORD`, `REQUIRE_API_KEY`, `APP_LOG_LEVEL` + +--- + +## Hard Rules (Never Violate) + +1. Never commit secrets or credentials +2. Never add logic to `localDb.ts` +3. Never use `eval()` / `new Function()` / implied eval +4. Never commit directly to `main` +5. Never write raw SQL in routes — use `src/lib/db/` modules +6. Never silently swallow errors in SSE streams +7. Always validate inputs with Zod schemas +8. Always include tests when changing production code +9. Coverage must stay ≥60% (statements, lines, functions, branches) diff --git a/open-sse/services/AGENTS.md b/open-sse/services/AGENTS.md new file mode 100644 index 00000000..9cfe947a --- /dev/null +++ b/open-sse/services/AGENTS.md @@ -0,0 +1,134 @@ +# open-sse/services/ — Routing Engine & Cross-Cutting Services + +**Purpose**: 36+ service modules powering request routing, rate limiting, quota management, token refresh, fallback strategies, and runtime state. The combo routing engine (`combo.ts`) is the core; supporting services handle resilience, accounting, and decision-making. + +--- + +## Key Services + +### Combo Routing Engine + +- **`combo.ts`** (800 LOC) — Entry point for multi-model routing. **`handleComboChat()`** iterates through targets in order until success or all fail. **`resolveComboTargets()`** expands combo config into ordered `ResolvedComboTarget[]` (provider + model + account + credentials). Enforces per-target circuit breaker and fallback logic. +- **Strategies** (13 total): `priority` (ordered list), `weighted` (probabilistic), `fill-first` (fill quota first), `round-robin`, `P2C` (power of two choices), `random`, `least-used`, `cost-optimized`, `strict-random`, `auto`, `lkgp` (last known good provider), `context-optimized`, `context-relay`. +- **Circuit Breaker**: Per target, tracks consecutive failures; breaks after threshold, reopens on success or timeout. + +### Quota & Rate Limiting + +- **`rateLimitManager.ts`** — Enforces upstream rate limits (429, retry-after headers). Implements token bucket per API key + provider combo. Rejects requests exceeding limits before dispatch. +- **`usage.ts`** — Tracks per-request token/cost consumption. Syncs with `quotaSnapshots` table. Reports cumulative usage for analytics. +- **`quotaCache.ts`** — In-memory quota snapshots. Invalidated on write; pre-loaded at startup. Prevents DB thrashing on high-volume requests. + +### Account & Token Management + +- **`tokenRefresh.ts`** — Handles OAuth token expiration. Detects 401 responses, triggers refresh via provider OAuth endpoint, retries request with new token. +- **`accountFallback.ts`** — If account reaches quota/rate-limit, switches to alternate account (combo targets). Logs account switch event. +- **`sessionManager.ts`** — Manages request session state across retries. Tracks session ID, attempt count, fallback history. + +### Request Routing & Intelligence + +- **`wildcardRouter.ts`** — Matches wildcard routes in combo configs (e.g., `gpt-*` → all GPT models). +- **`intentClassifier.ts`** — Classifies request intent (chat, embedding, image, video, etc.) for intelligent routing. +- **`taskAwareRouter.ts`** — Routes based on task characteristics (reasoning-heavy → o1, code-gen → Cursor, long-context → Claude). +- **`thinkingBudget.ts`** — Allocates thinking tokens for o1/o3 models; enforces per-request budget. +- **`contextManager.ts`** — Injects routing context (system prompts, memory) into requests. + +### Model Lifecycle & Fallback + +- **`modelDeprecation.ts`** — Detects deprecated models (gpt-3.5, claude-2, etc.). Routes to successor models automatically. +- **`modelFamilyFallback.ts`** — T5 intra-family fallback: if `gpt-4-turbo` unavailable, tries `gpt-4-1106-preview`, then `gpt-4`. +- **`emergencyFallback.ts`** — Last-resort fallback when all combo targets fail. Routes to stable free provider (Qwen Code, Gemini CLI fallback). + +### State & Detection + +- **`workflowFSM.ts`** — Finite state machine for multi-turn workflows (prompt engineering → execution → validation). +- **`backgroundTaskDetector.ts`** — Detects long-running background tasks; routes to batch APIs or defers execution. +- **`ipFilter.ts`** — IP-based routing rules (geographic or access control). +- **`signatureCache.ts`** — Caches request signatures for duplicate detection and deduplication. +- **`volumeDetector.ts`** — Detects request volume spikes; triggers rate-limit escalation or load-shedding. +- **`contextHandoff.ts`** — Serializes/restores session context for agent handoff (A2A protocol). + +### Auto-Routing & Adaptive + +- **`autoCombo/`** — Auto-generates combo configs based on historical performance, cost, and latency. +- **`modelFamilyFallback.ts`** — Automatic fallback within model families (T5, GPT-4, Claude). + +### Advanced Services + +- **`promptInjectionGuard.ts`** (middleware) — Clones request, sanitizes user input, detects prompt injection patterns before dispatch +- **`costRules.ts`** (domain layer) — Cost-based routing decisions (cheapest-first, within budget) +- **`degradation.ts`** (domain layer) — Handles service degradation scenarios (provider down, quota exceeded) +- **`resilience.ts`** — Retry logic, exponential backoff, circuit breaker orchestration across all services + +--- + +## Complexity Hotspots + +| Module | Lines | Risk | Mitigation | +| ------------------------ | ----- | ---------------------------------------------------------- | ------------------------------------------------------------------------------ | +| `combo.ts` | ~800 | High — routing logic, strategy dispatch, fallback ordering | Unit tests for each strategy, integration tests for combo sequences | +| `providerRegistry.ts` | 3000+ | High — 100+ provider configs, executor dispatch | Auto-validate via Zod at module load, split into provider-specific sub-modules | +| `rateLimitManager.ts` | ~300 | Medium — token bucket state, concurrent requests | Unit tests for bucket refill, edge cases (clock skew, parallel requests) | +| `modelFamilyFallback.ts` | ~200 | Medium — fallback chains, family detection | Test all family chains, ensure no circular fallbacks | + +--- + +## Testing Strategy + +Each service requires unit and integration tests. For authoritative coverage requirements and test execution guidelines, see [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests). + +- **Unit tests** — Each service in isolation with mocked dependencies (combos, models, executors) +- **Integration tests** — Combo routing with real combo configs, verify target resolution and fallback behavior +- **E2E tests** — Full request flow: chat → combo routing → provider selection → response streaming +- **Chaos tests** — Simulate provider failures, rate limits, token expiration; verify graceful degradation +- **Benchmarks** — Measure routing latency, combo resolution time (target: <10ms for 50 targets) + +--- + +## Performance Constraints + +- **Combo resolution**: <10ms for typical configs (5–20 targets) +- **Rate limit checks**: <1ms (in-memory token bucket) +- **Model family fallback**: <5ms (cached family definitions) +- **Request routing dispatch**: <2ms (hot path, pre-computed strategy dispatch) +- **No blocking I/O** in routing hot path — all async, no awaits on DB queries outside context injection + +--- + +## Anti-Patterns + +- ❌ Synchronous DB calls in `combo.ts` hot path — pre-compute and cache +- ❌ Retry logic in handlers; use `retry()` from resilience service +- ❌ Direct provider config access; use `providerRegistry` getter functions +- ❌ Hardcoded fallback chains; define in `modelFamilyFallback.ts` instead +- ❌ State mutations across concurrent requests; use request-scoped context only + +--- + +## Adding a New Service + +1. Create `open-sse/services/[serviceName].ts` with clear responsibilities +2. Export main handler function and any constants +3. Add unit tests in `tests/unit/services/[serviceName].test.mjs` +4. Integrate into request pipeline in `handlers/chatCore.ts` (if routing-related) or expose via combo.ts +5. Update routing logic in `combo.ts` if service affects target selection or fallback +6. Document in this file (table, key decisions section) + +--- + +## Key Decisions + +- **Combo-first design**: All routing decisions go through combo engine; fallback strategies are combo targets, not ad-hoc logic +- **Service composition**: Small focused modules; combo.ts orchestrates them, not monolithic routing +- **Circuit breaker per-target**: Failures isolated to specific provider+account combo; other targets unaffected +- **Caching everywhere**: Models, providers, quotas, family fallbacks all pre-cached; invalidated on write +- **13 strategies** over hardcoded logic: Strategy pattern allows new routing logic without touching combo.ts core + +--- + +## Review Focus + +- New services must not add blocking I/O to routing hot path +- Combo target resolution under 10ms (measure with benchmarks) +- Circuit breaker state per-target, not global +- All fallback chains tested (no infinite loops) +- Coverage requirements: See [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) (60% gate enforced in CI) diff --git a/src/lib/db/AGENTS.md b/src/lib/db/AGENTS.md new file mode 100644 index 00000000..6ea7c379 --- /dev/null +++ b/src/lib/db/AGENTS.md @@ -0,0 +1,102 @@ +# src/lib/db/ — SQLite Persistence Layer + +**Purpose**: Domain-driven SQLite persistence. Each module owns a specific table set. Schema migrations are versioned and idempotent. No raw SQL in routes — all ops go through `src/lib/db/` modules. + +--- + +## Key Modules + +### Core Infrastructure + +- **`core.ts`** — `getDbInstance()` returns singleton `better-sqlite3` with WAL journaling. Exports `rowToCamel()` (snake_case → camelCase), `encryptConnectionFields()` for provider credentials at rest. `SCHEMA_SQL` defines 15 base tables. +- **`migrationRunner.ts`** — Applies versioned SQL files from `db/migrations/` inside transactions. Tracks applied migrations in `_omniroute_migrations`. Runs at startup; each migration is idempotent. +- **`db/migrations/`** — 21 SQL files (`001_initial_schema.sql` → `021_combo_call_log_targets.sql`). Each migration has single responsibility, runs in a transaction, never fails partially. +- **`localDb.ts`** — Re-export layer only. Never add logic here. Consumers import domain modules from this file for convenience. + +### Domain Modules (22 total) + +Each module owns specific tables + CRUD operations: + +| Module | Tables | Responsibility | +| ----------------------- | ------------------------- | ------------------------------------------------------- | +| `providers.ts` | `provider_connections` | OAuth/API key provider registration and credentials | +| `models.ts` | `models` | Model definitions, capabilities, pricing | +| `combos.ts` | `combos`, `combo_targets` | Combo routing configs, target ordering | +| `apiKeys.ts` | `api_keys` | API key lifecycle, scopes, quota tracking | +| `settings.ts` | `settings` | KV store for system configuration | +| `backup.ts` | Backup export/import ops | Serialize/deserialize entire DB state | +| `proxies.ts` | `proxies` | MITM proxy configs and routing rules | +| `prompts.ts` | `prompts` | Reusable prompt templates, versioning | +| `webhooks.ts` | `webhooks` | Event-driven webhook subscriptions and logs | +| `detailedLogs.ts` | `detailed_logs` | Per-request audit logging (optional, high volume) | +| `domainState.ts` | `domain_state` | Transient runtime state (not persisted across restarts) | +| `registeredKeys.ts` | `registered_keys` | Whitelisted API keys for MCP/A2A access | +| `quotaSnapshots.ts` | `quota_snapshots` | Historical quota usage for analytics | +| `modelComboMappings.ts` | `model_combo_mappings` | Map models to combo defaults | +| `cliToolState.ts` | `cli_tool_state` | CLI-specific persistent state | +| `encryption.ts` | — | Helpers for encrypting/decrypting sensitive fields | +| `readCache.ts` | — | In-memory cache for read-heavy ops (models, providers) | +| `secrets.ts` | `secrets` | Encrypted secret storage (API keys at rest) | +| `stateReset.ts` | — | Wipe/reset DB state for testing or recovery | +| `contextHandoffs.ts` | `context_handoffs` | Store/retrieve session context for agent handoff | +| `migrations/` | — | Versioned SQL schema evolution | +| `core.ts` | — | Singleton DB instance, helpers, schema definition | + +### Encryption & Security + +- **Sensitive fields** (API keys, OAuth tokens, connection strings) encrypted at rest using `src/lib/encryption/` utilities +- **`encryptConnectionFields()`** in `core.ts` — Automatic encryption when storing provider credentials +- **`secrets.ts`** — Dedicated encrypted store for long-term secret handling +- **Never log** SQLite encryption keys or raw secrets; always use redacted values in logs + +### Testing Strategy + +For authoritative coverage requirements and test execution guidelines, see [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) (lines 136–162). + +- **Unit tests** mock `getDbInstance()` to return isolated sqlite in-memory instance +- **Integration tests** use real SQLite with migrations applied, data cleaned up after each test +- **No fixture interdependencies** — each test runs migrations fresh +- Test files: `tests/unit/db/*.test.mjs`, `tests/integration/db/*.test.mjs` + +### Anti-Patterns + +- ❌ Raw SQL in routes — always use domain module functions +- ❌ Direct `prepare()` statements outside `db/` modules — breaks modularity +- ❌ Mixing encryption logic in domain modules — use `encryption.ts` helpers only +- ❌ Accessing `provider_connections` table from `combos.ts` — each module owns its tables +- ❌ Skipping migrations for schema changes — all changes go through `db/migrations/` + +### Adding a New Domain Module + +1. Create `src/lib/db/[module].ts` with CRUD functions (create, read, update, delete, list) +2. Export from `src/lib/db/localDb.ts` (add re-export) +3. If new tables required: create migration in `db/migrations/NNN_[description].sql` +4. Run migration via `migrationRunner.ts` at startup (automatic) +5. Add unit tests in `tests/unit/db/[module].test.mjs` +6. Ensure tests meet the coverage requirements in [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) + +### Performance Notes + +- **Read cache** (`readCache.ts`) — Pre-loads frequently accessed data (models, providers) at startup; invalidated on write +- **WAL journaling** (`core.ts`) — Enables concurrent reads during writes +- **Batch operations** — Use prepared statements with parameter binding to avoid SQL injection +- **Connection pooling** — Singleton pattern prevents per-request connection overhead + +--- + +## Key Decisions + +- **SQLite over PostgreSQL**: Simpler deployment, no separate database server, encryption at application layer +- **Versioned migrations**: Each schema change is tracked, reproducible, reversible with effort +- **Domain modules**: Enforces single responsibility, prevents cross-module table access +- **Re-export layer**: Convenience for consumers; `localDb.ts` is re-export-only to prevent circular dependencies + +--- + +## Review Focus + +- DB module changes must preserve domain boundaries (one module = one table set) +- New migrations must be idempotent and run inside transactions +- Encryption helpers used for all sensitive fields +- Test coverage and PR requirements: see [`CONTRIBUTING.md#running-tests`](../../CONTRIBUTING.md#running-tests) (lines 136–162) +- No raw SQL in routes or non-db modules