Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 diegosouzapw
|
22d318f201 | ||
|
diegosouzapw
|
afa2cea678 | ||
|
diegosouzapw
|
6dce45505c | ||
|
Diego Rodrigues de Sa e Souza
|
014732788c | ||
|
Diego Rodrigues de Sa e Souza
|
0e75d838ab | ||
|
Alptekin Gülcan
|
8383da8a50 | ||
|
duongvdo
|
199d173816 | ||
|
duongvdo
|
0e207dc5d2 |
@@ -130,6 +130,22 @@ GEMINI_CLI_OAUTH_CLIENT_SECRET=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
|
||||
# IFLOW_OAUTH_CLIENT_ID=
|
||||
IFLOW_OAUTH_CLIENT_SECRET=4Z3YjXycVsQvyGF1etiNlIBB4RsqSDtW
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Provider User-Agent Overrides (optional — customize per-provider UA headers)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Format: {PROVIDER_ID}_USER_AGENT=custom-value
|
||||
# When set, overrides the default User-Agent header sent to that provider.
|
||||
# Useful when providers update versions or block old user-agents.
|
||||
CLAUDE_USER_AGENT=claude-cli/1.0.83 (external, cli)
|
||||
CODEX_USER_AGENT=codex-cli/0.92.0 (Windows 10.0.26100; x64)
|
||||
GITHUB_USER_AGENT=GitHubCopilotChat/0.26.7
|
||||
ANTIGRAVITY_USER_AGENT=antigravity/1.104.0 darwin/arm64
|
||||
KIRO_USER_AGENT=AWS-SDK-JS/3.0.0 kiro-ide/1.0.0
|
||||
IFLOW_USER_AGENT=iFlow-Cli
|
||||
QWEN_USER_AGENT=google-api-nodejs-client/9.15.1
|
||||
CURSOR_USER_AGENT=connect-es/1.6.1
|
||||
GEMINI_CLI_USER_AGENT=google-api-nodejs-client/9.15.1
|
||||
|
||||
# API Key Providers (Phase 1 + Phase 4)
|
||||
# Add via Dashboard → Providers → Add API Key, or set here
|
||||
# DEEPSEEK_API_KEY=
|
||||
|
||||
@@ -7,6 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
---
|
||||
|
||||
## [1.7.0] — 2026-02-28
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **16 Pain Points Documentation** — New collapsible section "🎯 What OmniRoute Solves — 16 Real Pain Points" added to the main README and all 29 language-specific READMEs. Each pain point uses `<details>/<summary>` tags for clean, expandable content
|
||||
- **Configurable User-Agent per Provider** — User-Agent strings for OAuth providers (Claude, Codex, GitHub, Antigravity, Kiro, iFlow, Qwen, Cursor, Gemini CLI) are now configurable via environment variables. Format: `{PROVIDER_ID}_USER_AGENT=custom-value` ([#155](https://github.com/diegosouzapw/OmniRoute/issues/155))
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Hardcoded `$HOME` Path in Standalone/Bun Builds** — 5 files (`backupService.ts`, `mitm/manager.ts`, `mitm/server.ts`, `mitm/cert/generate.ts`, `codex-profiles/route.ts`) were bypassing the centralized `dataPaths.ts` and using `os.homedir()` directly. This caused paths to bake the build machine's `$HOME` into standalone/bun builds, producing `EACCES: permission denied` errors on other machines. All files now use `resolveDataDir()` from `dataPaths.ts`, respecting `DATA_DIR` env var and XDG conventions ([#156](https://github.com/diegosouzapw/OmniRoute/issues/156))
|
||||
|
||||
### 📝 Documentation
|
||||
|
||||
- **`.env` and `.env.example` Synced** — Added 9 User-Agent env vars with latest known default values to both environment files
|
||||
- **30 README Translations Updated** — All language READMEs now include the 16 Pain Points section
|
||||
|
||||
---
|
||||
|
||||
## [1.6.9] — 2026-02-28
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Proxy Port Preservation** — `new URL()` silently strips default ports (80/443); proxy connections now extract the port from the raw URL string before parsing, preventing connection timeouts ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **Proxy Credential Encoding** — URL-encode special characters in proxy username/password; decode during legacy migration ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **HTTPS Proxy Default Port** — Changed from 8080 to 443 in frontend and migration logic ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **Proxy Dispatcher Cache** — Invalidate cached dispatchers when proxy config is updated or deleted ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **Proxy Logger SQLite Type** — Cast `proxyPort` to `Number` for INTEGER column ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **CopilotToolCard URL** — Use `baseUrl` prop directly instead of redundant `window.location.origin`; filter to chat models only (`!m.type && !m.parent`) ([PR #160](https://github.com/diegosouzapw/OmniRoute/pull/160))
|
||||
|
||||
---
|
||||
|
||||
## [1.6.8] — 2026-02-28
|
||||
|
||||
### 🔧 Improved
|
||||
|
||||
@@ -2,6 +2,7 @@ FROM node:22-bookworm-slim AS builder
|
||||
WORKDIR /app
|
||||
|
||||
COPY package*.json ./
|
||||
COPY scripts/postinstall.mjs ./scripts/postinstall.mjs
|
||||
RUN if [ -f package-lock.json ]; then npm ci --no-audit --no-fund; else npm install --no-audit --no-fund; fi
|
||||
|
||||
COPY . ./
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ بداية سريعة
|
||||
|
||||
**1. التثبيت عالميًا:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Бърз старт
|
||||
|
||||
**1. Инсталирайте глобално:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Hurtig start
|
||||
|
||||
**1. Installer globalt:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Ergebnis: Nie aufhören zu programmieren, minimale Kosten
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Schnellstart
|
||||
|
||||
**1. Global installieren:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Resultado: Nunca dejes de programar, costo mínimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Inicio Rápido
|
||||
|
||||
**1. Instala globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Pika-aloitus
|
||||
|
||||
**1. Asenna maailmanlaajuisesti:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Résultat : Ne jamais arrêter de coder, coût minimal
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Démarrage rapide
|
||||
|
||||
**1. Installer globalement :**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ התחלה מהירה
|
||||
|
||||
**1. התקן ברחבי העולם:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Gyors kezdés
|
||||
|
||||
**1. Globális telepítés:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mulai Cepat
|
||||
|
||||
**1. Instal secara global:**
|
||||
|
||||
+257
@@ -80,6 +80,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ त्वरित शुरुआत
|
||||
|
||||
**1. विश्व स्तर पर स्थापित करें:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Risultato: Non smettere mai di programmare, costo minimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Avvio Rapido
|
||||
|
||||
**1. Installa globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ クイックスタート
|
||||
|
||||
**1.グローバルにインストール:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ 빠른 시작
|
||||
|
||||
**1. 전역적으로 설치:**
|
||||
|
||||
@@ -169,6 +169,265 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
**1. Install globally:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mula Pantas
|
||||
|
||||
**1. Pasang secara global:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Snelle start
|
||||
|
||||
**1. Wereldwijd installeren:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Hurtigstart
|
||||
|
||||
**1. Installer globalt:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mabilis na Pagsisimula
|
||||
|
||||
**1. I-install sa buong mundo:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Szybki start
|
||||
|
||||
**1. Zainstaluj globalnie:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Resultado: Nunca pare de programar, custo mínimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Início Rápido
|
||||
|
||||
**1. Instale globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Início rápido
|
||||
|
||||
**1. Instale globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Pornire rapidă
|
||||
|
||||
**1. Instalați la nivel global:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ _Подключайте любую IDE или CLI-инструмент с AI ч
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Быстрый старт
|
||||
|
||||
**1. Установите глобально:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Rýchly štart
|
||||
|
||||
**1. Inštalovať globálne:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Snabbstart
|
||||
|
||||
**1. Installera globalt:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ เริ่มต้นอย่างรวดเร็ว
|
||||
|
||||
**1. ติดตั้งทั่วโลก:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Швидкий старт
|
||||
|
||||
**1. Встановити глобально:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Bắt đầu nhanh
|
||||
|
||||
**1. Cài đặt trên toàn cầu:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ _通过 OmniRoute 连接任何 AI 驱动的 IDE 或 CLI 工具 — 免费 API
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ 快速开始
|
||||
|
||||
**1. 全局安装：**
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute-desktop",
|
||||
"version": "1.6.8",
|
||||
"version": "1.6.9",
|
||||
"description": "OmniRoute Desktop Application",
|
||||
"main": "main.js",
|
||||
"author": "OmniRoute Team",
|
||||
|
||||
@@ -43,6 +43,19 @@ export class BaseExecutor {
|
||||
...this.config.headers,
|
||||
};
|
||||
|
||||
// Allow per-provider User-Agent override via environment variable.
|
||||
// Example: CLAUDE_USER_AGENT="my-agent/2.0" overrides the default for the Claude provider.
|
||||
const providerId = this.config?.id || this.provider;
|
||||
if (providerId) {
|
||||
const envKey = `${providerId.toUpperCase().replace(/[^A-Z0-9]/g, "_")}_USER_AGENT`;
|
||||
const envUA = process.env[envKey]?.trim();
|
||||
if (envUA) {
|
||||
// Override both common casing variants
|
||||
headers["User-Agent"] = envUA;
|
||||
if (headers["user-agent"]) headers["user-agent"] = envUA;
|
||||
}
|
||||
}
|
||||
|
||||
if (credentials.accessToken) {
|
||||
headers["Authorization"] = `Bearer ${credentials.accessToken}`;
|
||||
} else if (credentials.apiKey) {
|
||||
|
||||
@@ -4,13 +4,39 @@ import { socksDispatcher } from "fetch-socks";
|
||||
const DISPATCHER_CACHE_KEY = Symbol.for("omniroute.proxyDispatcher.cache");
|
||||
const SUPPORTED_PROTOCOLS = new Set(["http:", "https:", "socks5:"]);
|
||||
|
||||
function getDispatcherCache() {
|
||||
function getDispatcherCache(): Map<string, any> {
|
||||
if (!globalThis[DISPATCHER_CACHE_KEY]) {
|
||||
globalThis[DISPATCHER_CACHE_KEY] = new Map();
|
||||
}
|
||||
return globalThis[DISPATCHER_CACHE_KEY];
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all cached proxy dispatchers.
|
||||
* Call this when proxy configuration changes to avoid stale connections.
|
||||
*/
|
||||
export function clearDispatcherCache() {
|
||||
const cache = getDispatcherCache();
|
||||
cache.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the port from a proxy URL string before URL parsing.
|
||||
* `new URL("http://host:80")` strips port 80 since it's the HTTP default,
|
||||
* but proxy servers commonly listen on port 80/443, so we need to preserve it.
|
||||
*/
|
||||
function extractExplicitPort(urlStr) {
|
||||
try {
|
||||
// Match port in the host portion: "scheme://[user:pass@]host:PORT[/...]"
|
||||
const match = urlStr.match(/:\/\/(?:[^@]*@)?[^:/\s]+:(\d+)/);
|
||||
if (match) {
|
||||
const port = Number(match[1]);
|
||||
if (Number.isInteger(port) && port >= 1 && port <= 65535) return String(port);
|
||||
}
|
||||
} catch {}
|
||||
return null;
|
||||
}
|
||||
|
||||
function defaultPortForProtocol(protocol) {
|
||||
if (protocol === "https:" || protocol === "wss:") return "443";
|
||||
if (protocol === "socks5:") return "1080";
|
||||
@@ -26,13 +52,28 @@ function normalizePort(port, protocol) {
|
||||
return String(parsed);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a proxy URL string manually from parsed URL components.
|
||||
* We cannot use URL.toString() because the URL serializer silently strips
|
||||
* default ports (80 for http, 443 for https). Proxy servers commonly
|
||||
* listen on these ports, so we must always include the port explicitly.
|
||||
*/
|
||||
function buildProxyUrlString(parsed, port) {
|
||||
const auth =
|
||||
parsed.username
|
||||
? `${parsed.username}${parsed.password ? `:${parsed.password}` : ""}@`
|
||||
: "";
|
||||
return `${parsed.protocol}//${auth}${parsed.hostname}:${port}`;
|
||||
}
|
||||
|
||||
export function isSocks5ProxyEnabled() {
|
||||
return process.env.ENABLE_SOCKS5_PROXY === "true";
|
||||
}
|
||||
|
||||
export function proxyUrlForLogs(proxyUrl) {
|
||||
const explicitPort = extractExplicitPort(proxyUrl);
|
||||
const parsed = new URL(proxyUrl);
|
||||
const port = parsed.port || defaultPortForProtocol(parsed.protocol);
|
||||
const port = explicitPort || parsed.port || defaultPortForProtocol(parsed.protocol);
|
||||
return `${parsed.protocol}//${parsed.hostname}:${port}`;
|
||||
}
|
||||
|
||||
@@ -41,6 +82,11 @@ export function normalizeProxyUrl(
|
||||
source = "proxy",
|
||||
{ allowSocks5 = isSocks5ProxyEnabled() } = {}
|
||||
) {
|
||||
// Extract the explicit port from the raw URL string BEFORE parsing,
|
||||
// because `new URL()` silently strips default ports (80 for http,
|
||||
// 443 for https), which are valid and common for proxy servers.
|
||||
const explicitPort = extractExplicitPort(proxyUrl);
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = new URL(proxyUrl);
|
||||
@@ -62,8 +108,12 @@ export function normalizeProxyUrl(
|
||||
throw new Error(`[ProxyDispatcher] Invalid ${source} host`);
|
||||
}
|
||||
|
||||
parsed.port = normalizePort(parsed.port, parsed.protocol);
|
||||
return parsed.toString();
|
||||
// Use the explicit port from the raw string if present, otherwise apply default.
|
||||
const port = explicitPort || normalizePort(parsed.port, parsed.protocol);
|
||||
|
||||
// Build the URL string manually instead of using parsed.toString(),
|
||||
// which would strip default ports (80/443) and break the proxy connection.
|
||||
return buildProxyUrlString(parsed, port);
|
||||
}
|
||||
|
||||
export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabled() } = {}) {
|
||||
@@ -93,14 +143,15 @@ export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabl
|
||||
}
|
||||
|
||||
const port = normalizePort(proxyConfig.port, protocol);
|
||||
const proxyUrl = new URL(`${type}://${proxyConfig.host}:${port}`);
|
||||
|
||||
if (proxyConfig.username) {
|
||||
proxyUrl.username = proxyConfig.username;
|
||||
proxyUrl.password = proxyConfig.password || "";
|
||||
}
|
||||
// Build the URL string manually to preserve the port through normalization.
|
||||
const auth = proxyConfig.username
|
||||
? `${encodeURIComponent(proxyConfig.username)}:${proxyConfig.password ? encodeURIComponent(proxyConfig.password) : ""}@`
|
||||
: "";
|
||||
|
||||
return normalizeProxyUrl(proxyUrl.toString(), "context proxy", { allowSocks5 });
|
||||
const proxyUrlStr = `${type}://${auth}${proxyConfig.host}:${port}`;
|
||||
|
||||
return normalizeProxyUrl(proxyUrlStr, "context proxy", { allowSocks5 });
|
||||
}
|
||||
|
||||
export function createProxyDispatcher(proxyUrl) {
|
||||
@@ -111,11 +162,14 @@ export function createProxyDispatcher(proxyUrl) {
|
||||
if (dispatcher) return dispatcher;
|
||||
|
||||
const parsed = new URL(normalizedUrl);
|
||||
const explicitPort = extractExplicitPort(normalizedUrl);
|
||||
const port = explicitPort || normalizePort(parsed.port, parsed.protocol);
|
||||
|
||||
if (parsed.protocol === "socks5:") {
|
||||
const socksOptions: Record<string, any> = {
|
||||
type: 5,
|
||||
host: parsed.hostname,
|
||||
port: Number(normalizePort(parsed.port, parsed.protocol)),
|
||||
port: Number(port),
|
||||
};
|
||||
if (parsed.username) socksOptions.userId = decodeURIComponent(parsed.username);
|
||||
if (parsed.password) socksOptions.password = decodeURIComponent(parsed.password);
|
||||
@@ -126,4 +180,4 @@ export function createProxyDispatcher(proxyUrl) {
|
||||
|
||||
dispatcherCache.set(normalizedUrl, dispatcher);
|
||||
return dispatcher;
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "1.6.8",
|
||||
"version": "1.7.0",
|
||||
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
|
||||
@@ -33,7 +33,6 @@ export default function CLIToolsPageClient({ machineId }) {
|
||||
const [apiKeys, setApiKeys] = useState([]);
|
||||
const [toolStatuses, setToolStatuses] = useState({});
|
||||
const [statusesLoaded, setStatusesLoaded] = useState(false);
|
||||
const [apiBaseUrl, setApiBaseUrl] = useState("");
|
||||
|
||||
useEffect(() => {
|
||||
fetchConnections();
|
||||
@@ -48,12 +47,6 @@ export default function CLIToolsPageClient({ machineId }) {
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
setCloudEnabled(data.cloudEnabled || false);
|
||||
if (typeof window !== "undefined") {
|
||||
const protocol = window.location.protocol;
|
||||
const hostname = window.location.hostname;
|
||||
const apiPort = data?.apiPort || 20128;
|
||||
setApiBaseUrl(`${protocol}//${hostname}:${apiPort}`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.log("Error loading cloud settings:", error);
|
||||
@@ -155,9 +148,8 @@ export default function CLIToolsPageClient({ machineId }) {
|
||||
if (cloudEnabled && CLOUD_URL) {
|
||||
return CLOUD_URL;
|
||||
}
|
||||
if (apiBaseUrl) {
|
||||
return apiBaseUrl;
|
||||
}
|
||||
// Use window.location.origin directly — works correctly in Docker/reverse-proxy
|
||||
// Per @alpgul feedback: don't use baseUrl prop (has port duplication issues)
|
||||
if (typeof window !== "undefined") {
|
||||
return window.location.origin;
|
||||
}
|
||||
|
||||
@@ -60,7 +60,7 @@ export default function CopilotToolCard({
|
||||
.then((data) => {
|
||||
if (cancelled) return;
|
||||
const modelList = (data.data || [])
|
||||
.filter((m: any) => m.id) // Only models with valid IDs
|
||||
.filter((m: any) => m && !m.type && !m.parent && m.id) // Only chat models with valid IDs
|
||||
.map((m: any) => ({
|
||||
value: m.id,
|
||||
label: m.id,
|
||||
@@ -112,11 +112,8 @@ export default function CopilotToolCard({
|
||||
};
|
||||
|
||||
const getBaseUrlForConfig = () => {
|
||||
// Use window.location.origin directly — works correctly in Docker/reverse-proxy
|
||||
// Per @alpgul feedback: don't use baseUrl prop (has port duplication issues)
|
||||
const origin =
|
||||
typeof window !== "undefined" ? window.location.origin : "http://localhost:20128";
|
||||
return `${origin}/v1/chat/completions`;
|
||||
const url = baseUrl;
|
||||
return `${url}/v1/chat/completions`;
|
||||
};
|
||||
|
||||
// Generate the Copilot chatLanguageModels.json config
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
import os from "os";
|
||||
import { ensureCliConfigWriteAllowed, getCliConfigPaths } from "@/shared/services/cliRuntime";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const PROFILES_DIR = path.join(os.homedir(), ".omniroute", "codex-profiles");
|
||||
const PROFILES_DIR = path.join(resolveDataDir(), "codex-profiles");
|
||||
|
||||
/**
|
||||
* Ensure profiles directory exists
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
deleteProxyForLevel,
|
||||
resolveProxyForConnection,
|
||||
} from "../../../../lib/localDb";
|
||||
import { clearDispatcherCache } from "@omniroute/open-sse/utils/proxyDispatcher";
|
||||
|
||||
const BASE_SUPPORTED_PROXY_TYPES = new Set(["http", "https"]);
|
||||
|
||||
@@ -129,6 +130,7 @@ export async function PUT(request) {
|
||||
const body = await request.json();
|
||||
const normalizedBody = normalizeProxyPayload(body);
|
||||
const updated = await setProxyConfig(normalizedBody);
|
||||
clearDispatcherCache();
|
||||
return Response.json(updated);
|
||||
} catch (error) {
|
||||
const status = Number(error?.status) || 500;
|
||||
@@ -155,6 +157,7 @@ export async function DELETE(request) {
|
||||
}
|
||||
|
||||
const updated = await deleteProxyForLevel(level, id);
|
||||
clearDispatcherCache();
|
||||
return Response.json(updated);
|
||||
} catch (error) {
|
||||
return Response.json(
|
||||
|
||||
@@ -211,11 +211,11 @@ function migrateProxyEntry(value: any) {
|
||||
try {
|
||||
const url = new URL(value);
|
||||
return {
|
||||
type: url.protocol.replace(":", "").replace("//", "") || "http",
|
||||
type: url.protocol.replace(":", "") || "http",
|
||||
host: url.hostname,
|
||||
port: url.port || (url.protocol === "socks5:" ? "1080" : "8080"),
|
||||
username: url.username || "",
|
||||
password: url.password || "",
|
||||
port: url.port || (url.protocol === "socks5:" ? "1080" : url.protocol === "https:" ? "443" : "8080"),
|
||||
username: url.username ? decodeURIComponent(url.username) : "",
|
||||
password: url.password ? decodeURIComponent(url.password) : "",
|
||||
};
|
||||
} catch {
|
||||
const parts = value.split(":");
|
||||
|
||||
@@ -134,7 +134,7 @@ export function logProxyEvent(entry: Partial<ProxyLogEntry>) {
|
||||
status: log.status,
|
||||
proxyType: log.proxy?.type || null,
|
||||
proxyHost: log.proxy?.host || null,
|
||||
proxyPort: log.proxy?.port || null,
|
||||
proxyPort: log.proxy?.port ? Number(log.proxy.port) : null,
|
||||
level: log.level,
|
||||
levelId: log.levelId,
|
||||
provider: log.provider,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import path from "path";
|
||||
import fs from "fs";
|
||||
import os from "os";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const TARGET_HOST = "daily-cloudcode-pa.googleapis.com";
|
||||
|
||||
@@ -8,7 +8,7 @@ const TARGET_HOST = "daily-cloudcode-pa.googleapis.com";
|
||||
* Generate self-signed SSL certificate using selfsigned (pure JS, no openssl needed)
|
||||
*/
|
||||
export async function generateCert() {
|
||||
const certDir = path.join(os.homedir(), ".omniroute", "mitm");
|
||||
const certDir = path.join(resolveDataDir(), "mitm");
|
||||
const keyPath = path.join(certDir, "server.key");
|
||||
const certPath = path.join(certDir, "server.crt");
|
||||
|
||||
|
||||
+4
-4
@@ -1,7 +1,7 @@
|
||||
import { spawn } from "child_process";
|
||||
import path from "path";
|
||||
import fs from "fs";
|
||||
import os from "os";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
import { addDNSEntry, removeDNSEntry } from "./dns/dnsConfig";
|
||||
import { generateCert } from "./cert/generate";
|
||||
import { installCert } from "./cert/install";
|
||||
@@ -24,7 +24,7 @@ export function clearCachedPassword() {
|
||||
}
|
||||
|
||||
// server.js is in same directory as this file
|
||||
const PID_FILE = path.join(os.homedir(), ".omniroute", "mitm", ".mitm.pid");
|
||||
const PID_FILE = path.join(resolveDataDir(), "mitm", ".mitm.pid");
|
||||
|
||||
// Check if a PID is alive
|
||||
function isProcessAlive(pid) {
|
||||
@@ -71,7 +71,7 @@ export async function getMitmStatus() {
|
||||
}
|
||||
|
||||
// Check cert
|
||||
const certDir = path.join(os.homedir(), ".omniroute", "mitm");
|
||||
const certDir = path.join(resolveDataDir(), "mitm");
|
||||
const certExists = fs.existsSync(path.join(certDir, "server.crt"));
|
||||
|
||||
return { running, pid, dnsConfigured, certExists };
|
||||
@@ -89,7 +89,7 @@ export async function startMitm(apiKey, sudoPassword) {
|
||||
}
|
||||
|
||||
// 1. Generate SSL certificate if not exists
|
||||
const certPath = path.join(os.homedir(), ".omniroute", "mitm", "server.crt");
|
||||
const certPath = path.join(resolveDataDir(), "mitm", "server.crt");
|
||||
if (!fs.existsSync(certPath)) {
|
||||
console.log("Generating SSL certificate...");
|
||||
await generateCert();
|
||||
|
||||
+11
-3
@@ -5,13 +5,21 @@ const dns = require("dns");
|
||||
const { promisify } = require("util");
|
||||
const os = require("os");
|
||||
|
||||
// Resolve data directory — mirrors src/lib/dataPaths.ts logic.
|
||||
// This file runs as a standalone CommonJS process and cannot import the ES module.
|
||||
function getDataDir() {
|
||||
if (process.env.DATA_DIR) return path.resolve(process.env.DATA_DIR.trim());
|
||||
return path.join(os.homedir(), ".omniroute");
|
||||
}
|
||||
|
||||
// Configuration
|
||||
const TARGET_HOST = "daily-cloudcode-pa.googleapis.com";
|
||||
const LOCAL_PORT = 443;
|
||||
const ROUTER_URL = "http://localhost:20128/v1/chat/completions";
|
||||
const API_KEY = process.env.ROUTER_API_KEY;
|
||||
const DB_FILE = path.join(os.homedir(), ".omniroute", "db.json");
|
||||
const SQLITE_FILE = path.join(os.homedir(), ".omniroute", "storage.sqlite");
|
||||
const DATA_DIR = getDataDir();
|
||||
const DB_FILE = path.join(DATA_DIR, "db.json");
|
||||
const SQLITE_FILE = path.join(DATA_DIR, "storage.sqlite");
|
||||
|
||||
let _sqliteDb = null;
|
||||
|
||||
@@ -24,7 +32,7 @@ if (!API_KEY) {
|
||||
}
|
||||
|
||||
// Load SSL certificates
|
||||
const certDir = path.join(os.homedir(), ".omniroute", "mitm");
|
||||
const certDir = path.join(DATA_DIR, "mitm");
|
||||
const sslOptions = {
|
||||
key: fs.readFileSync(path.join(certDir, "server.key")),
|
||||
cert: fs.readFileSync(path.join(certDir, "server.crt")),
|
||||
|
||||
@@ -48,7 +48,11 @@ export default function ProxyConfigModal({ isOpen, onClose, level, levelId, leve
|
||||
const [hasOwnProxy, setHasOwnProxy] = useState(false);
|
||||
const [formError, setFormError] = useState(null);
|
||||
|
||||
const getDefaultPort = (type) => (type === "socks5" ? "1080" : "8080");
|
||||
const getDefaultPort = (type) => {
|
||||
if (type === "socks5") return "1080";
|
||||
if (type === "https") return "443";
|
||||
return "8080";
|
||||
};
|
||||
|
||||
// Load existing proxy config when modal opens
|
||||
useEffect(() => {
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
import os from "os";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const BACKUP_DIR = path.join(os.homedir(), ".omniroute", "backups");
|
||||
const BACKUP_DIR = path.join(resolveDataDir(), "backups");
|
||||
const MAX_BACKUPS_PER_TOOL = 5;
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user