- Add collapsible '16 Real Pain Points' section to all 30 READMEs - Fix 5 files bypassing dataPaths.ts with hardcoded os.homedir() (closes #156) - Add per-provider User-Agent env var overrides in base executor (closes #155) - Sync .env and .env.example with 9 provider UA defaults - Update CHANGELOG.md for v1.7.0
This commit is contained in:
diegosouzapw
@@ -130,6 +130,22 @@ GEMINI_CLI_OAUTH_CLIENT_SECRET=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
|
||||
# IFLOW_OAUTH_CLIENT_ID=
|
||||
IFLOW_OAUTH_CLIENT_SECRET=4Z3YjXycVsQvyGF1etiNlIBB4RsqSDtW
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Provider User-Agent Overrides (optional — customize per-provider UA headers)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Format: {PROVIDER_ID}_USER_AGENT=custom-value
|
||||
# When set, overrides the default User-Agent header sent to that provider.
|
||||
# Useful when providers update versions or block old user-agents.
|
||||
CLAUDE_USER_AGENT=claude-cli/1.0.83 (external, cli)
|
||||
CODEX_USER_AGENT=codex-cli/0.92.0 (Windows 10.0.26100; x64)
|
||||
GITHUB_USER_AGENT=GitHubCopilotChat/0.26.7
|
||||
ANTIGRAVITY_USER_AGENT=antigravity/1.104.0 darwin/arm64
|
||||
KIRO_USER_AGENT=AWS-SDK-JS/3.0.0 kiro-ide/1.0.0
|
||||
IFLOW_USER_AGENT=iFlow-Cli
|
||||
QWEN_USER_AGENT=google-api-nodejs-client/9.15.1
|
||||
CURSOR_USER_AGENT=connect-es/1.6.1
|
||||
GEMINI_CLI_USER_AGENT=google-api-nodejs-client/9.15.1
|
||||
|
||||
# API Key Providers (Phase 1 + Phase 4)
|
||||
# Add via Dashboard → Providers → Add API Key, or set here
|
||||
# DEEPSEEK_API_KEY=
|
||||
|
||||
@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
---
|
||||
|
||||
## [1.7.0] — 2026-02-28
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **16 Pain Points Documentation** — New collapsible section "🎯 What OmniRoute Solves — 16 Real Pain Points" added to the main README and all 29 language-specific READMEs. Each pain point uses `<details>/<summary>` tags for clean, expandable content
|
||||
- **Configurable User-Agent per Provider** — User-Agent strings for OAuth providers (Claude, Codex, GitHub, Antigravity, Kiro, iFlow, Qwen, Cursor, Gemini CLI) are now configurable via environment variables. Format: `{PROVIDER_ID}_USER_AGENT=custom-value` ([#155](https://github.com/diegosouzapw/OmniRoute/issues/155))
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Hardcoded `$HOME` Path in Standalone/Bun Builds** — 5 files (`backupService.ts`, `mitm/manager.ts`, `mitm/server.ts`, `mitm/cert/generate.ts`, `codex-profiles/route.ts`) were bypassing the centralized `dataPaths.ts` and using `os.homedir()` directly. This caused paths to bake the build machine's `$HOME` into standalone/bun builds, producing `EACCES: permission denied` errors on other machines. All files now use `resolveDataDir()` from `dataPaths.ts`, respecting `DATA_DIR` env var and XDG conventions ([#156](https://github.com/diegosouzapw/OmniRoute/issues/156))
|
||||
|
||||
### 📝 Documentation
|
||||
|
||||
- **`.env` and `.env.example` Synced** — Added 9 User-Agent env vars with latest known default values to both environment files
|
||||
- **30 README Translations Updated** — All language READMEs now include the 16 Pain Points section
|
||||
|
||||
---
|
||||
|
||||
## [1.6.9] — 2026-02-28
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ بداية سريعة
|
||||
|
||||
**1. التثبيت عالميًا:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Бърз старт
|
||||
|
||||
**1. Инсталирайте глобално:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Hurtig start
|
||||
|
||||
**1. Installer globalt:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Ergebnis: Nie aufhören zu programmieren, minimale Kosten
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Schnellstart
|
||||
|
||||
**1. Global installieren:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Resultado: Nunca dejes de programar, costo mínimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Inicio Rápido
|
||||
|
||||
**1. Instala globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Pika-aloitus
|
||||
|
||||
**1. Asenna maailmanlaajuisesti:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Résultat : Ne jamais arrêter de coder, coût minimal
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Démarrage rapide
|
||||
|
||||
**1. Installer globalement :**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ התחלה מהירה
|
||||
|
||||
**1. התקן ברחבי העולם:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Gyors kezdés
|
||||
|
||||
**1. Globális telepítés:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mulai Cepat
|
||||
|
||||
**1. Instal secara global:**
|
||||
|
||||
+257
@@ -80,6 +80,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ त्वरित शुरुआत
|
||||
|
||||
**1. विश्व स्तर पर स्थापित करें:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Risultato: Non smettere mai di programmare, costo minimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Avvio Rapido
|
||||
|
||||
**1. Installa globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ クイックスタート
|
||||
|
||||
**1.グローバルにインストール:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ 빠른 시작
|
||||
|
||||
**1. 전역적으로 설치:**
|
||||
|
||||
@@ -169,6 +169,265 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
**1. Install globally:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mula Pantas
|
||||
|
||||
**1. Pasang secara global:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Snelle start
|
||||
|
||||
**1. Wereldwijd installeren:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Hurtigstart
|
||||
|
||||
**1. Installer globalt:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mabilis na Pagsisimula
|
||||
|
||||
**1. I-install sa buong mundo:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Szybki start
|
||||
|
||||
**1. Zainstaluj globalnie:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Resultado: Nunca pare de programar, custo mínimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Início Rápido
|
||||
|
||||
**1. Instale globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Início rápido
|
||||
|
||||
**1. Instale globalmente:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Pornire rapidă
|
||||
|
||||
**1. Instalați la nivel global:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ _Подключайте любую IDE или CLI-инструмент с AI ч
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Быстрый старт
|
||||
|
||||
**1. Установите глобально:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Rýchly štart
|
||||
|
||||
**1. Inštalovať globálne:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Snabbstart
|
||||
|
||||
**1. Installera globalt:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ เริ่มต้นอย่างรวดเร็ว
|
||||
|
||||
**1. ติดตั้งทั่วโลก:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Швидкий старт
|
||||
|
||||
**1. Встановити глобально:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Bắt đầu nhanh
|
||||
|
||||
**1. Cài đặt trên toàn cầu:**
|
||||
|
||||
+257
@@ -157,6 +157,263 @@ _通过 OmniRoute 连接任何 AI 驱动的 IDE 或 CLI 工具 — 免费 API
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ 快速开始
|
||||
|
||||
**1. 全局安装：**
|
||||
|
||||
@@ -43,6 +43,19 @@ export class BaseExecutor {
|
||||
...this.config.headers,
|
||||
};
|
||||
|
||||
// Allow per-provider User-Agent override via environment variable.
|
||||
// Example: CLAUDE_USER_AGENT="my-agent/2.0" overrides the default for the Claude provider.
|
||||
const providerId = this.config?.id || this.provider;
|
||||
if (providerId) {
|
||||
const envKey = `${providerId.toUpperCase().replace(/[^A-Z0-9]/g, "_")}_USER_AGENT`;
|
||||
const envUA = process.env[envKey]?.trim();
|
||||
if (envUA) {
|
||||
// Override both common casing variants
|
||||
headers["User-Agent"] = envUA;
|
||||
if (headers["user-agent"]) headers["user-agent"] = envUA;
|
||||
}
|
||||
}
|
||||
|
||||
if (credentials.accessToken) {
|
||||
headers["Authorization"] = `Bearer ${credentials.accessToken}`;
|
||||
} else if (credentials.apiKey) {
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
import os from "os";
|
||||
import { ensureCliConfigWriteAllowed, getCliConfigPaths } from "@/shared/services/cliRuntime";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const PROFILES_DIR = path.join(os.homedir(), ".omniroute", "codex-profiles");
|
||||
const PROFILES_DIR = path.join(resolveDataDir(), "codex-profiles");
|
||||
|
||||
/**
|
||||
* Ensure profiles directory exists
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import path from "path";
|
||||
import fs from "fs";
|
||||
import os from "os";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const TARGET_HOST = "daily-cloudcode-pa.googleapis.com";
|
||||
|
||||
@@ -8,7 +8,7 @@ const TARGET_HOST = "daily-cloudcode-pa.googleapis.com";
|
||||
* Generate self-signed SSL certificate using selfsigned (pure JS, no openssl needed)
|
||||
*/
|
||||
export async function generateCert() {
|
||||
const certDir = path.join(os.homedir(), ".omniroute", "mitm");
|
||||
const certDir = path.join(resolveDataDir(), "mitm");
|
||||
const keyPath = path.join(certDir, "server.key");
|
||||
const certPath = path.join(certDir, "server.crt");
|
||||
|
||||
|
||||
+4
-4
@@ -1,7 +1,7 @@
|
||||
import { spawn } from "child_process";
|
||||
import path from "path";
|
||||
import fs from "fs";
|
||||
import os from "os";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
import { addDNSEntry, removeDNSEntry } from "./dns/dnsConfig";
|
||||
import { generateCert } from "./cert/generate";
|
||||
import { installCert } from "./cert/install";
|
||||
@@ -24,7 +24,7 @@ export function clearCachedPassword() {
|
||||
}
|
||||
|
||||
// server.js is in same directory as this file
|
||||
const PID_FILE = path.join(os.homedir(), ".omniroute", "mitm", ".mitm.pid");
|
||||
const PID_FILE = path.join(resolveDataDir(), "mitm", ".mitm.pid");
|
||||
|
||||
// Check if a PID is alive
|
||||
function isProcessAlive(pid) {
|
||||
@@ -71,7 +71,7 @@ export async function getMitmStatus() {
|
||||
}
|
||||
|
||||
// Check cert
|
||||
const certDir = path.join(os.homedir(), ".omniroute", "mitm");
|
||||
const certDir = path.join(resolveDataDir(), "mitm");
|
||||
const certExists = fs.existsSync(path.join(certDir, "server.crt"));
|
||||
|
||||
return { running, pid, dnsConfigured, certExists };
|
||||
@@ -89,7 +89,7 @@ export async function startMitm(apiKey, sudoPassword) {
|
||||
}
|
||||
|
||||
// 1. Generate SSL certificate if not exists
|
||||
const certPath = path.join(os.homedir(), ".omniroute", "mitm", "server.crt");
|
||||
const certPath = path.join(resolveDataDir(), "mitm", "server.crt");
|
||||
if (!fs.existsSync(certPath)) {
|
||||
console.log("Generating SSL certificate...");
|
||||
await generateCert();
|
||||
|
||||
+11
-3
@@ -5,13 +5,21 @@ const dns = require("dns");
|
||||
const { promisify } = require("util");
|
||||
const os = require("os");
|
||||
|
||||
// Resolve data directory — mirrors src/lib/dataPaths.ts logic.
|
||||
// This file runs as a standalone CommonJS process and cannot import the ES module.
|
||||
function getDataDir() {
|
||||
if (process.env.DATA_DIR) return path.resolve(process.env.DATA_DIR.trim());
|
||||
return path.join(os.homedir(), ".omniroute");
|
||||
}
|
||||
|
||||
// Configuration
|
||||
const TARGET_HOST = "daily-cloudcode-pa.googleapis.com";
|
||||
const LOCAL_PORT = 443;
|
||||
const ROUTER_URL = "http://localhost:20128/v1/chat/completions";
|
||||
const API_KEY = process.env.ROUTER_API_KEY;
|
||||
const DB_FILE = path.join(os.homedir(), ".omniroute", "db.json");
|
||||
const SQLITE_FILE = path.join(os.homedir(), ".omniroute", "storage.sqlite");
|
||||
const DATA_DIR = getDataDir();
|
||||
const DB_FILE = path.join(DATA_DIR, "db.json");
|
||||
const SQLITE_FILE = path.join(DATA_DIR, "storage.sqlite");
|
||||
|
||||
let _sqliteDb = null;
|
||||
|
||||
@@ -24,7 +32,7 @@ if (!API_KEY) {
|
||||
}
|
||||
|
||||
// Load SSL certificates
|
||||
const certDir = path.join(os.homedir(), ".omniroute", "mitm");
|
||||
const certDir = path.join(DATA_DIR, "mitm");
|
||||
const sslOptions = {
|
||||
key: fs.readFileSync(path.join(certDir, "server.key")),
|
||||
cert: fs.readFileSync(path.join(certDir, "server.crt")),
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
import os from "os";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const BACKUP_DIR = path.join(os.homedir(), ".omniroute", "backups");
|
||||
const BACKUP_DIR = path.join(resolveDataDir(), "backups");
|
||||
const MAX_BACKUPS_PER_TOOL = 5;
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user