Compare commits
57 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2902a0fe26 | |||
| 7ba57634c1 | |||
| 211dde25d0 | |||
| 57ff59aef2 | |||
| c39faba2b5 | |||
| 212bca2e1e | |||
| f807c56e31 | |||
| 5510c25040 | |||
| 9d884d2d60 | |||
| f26fa67374 | |||
| ccb314e065 | |||
| 0517dcf0b7 | |||
| b7f0665ce9 | |||
| 144628755d | |||
| 7c5bb2c6b6 | |||
| afadb0fea1 | |||
| b6c9c8a822 | |||
| 11f43ca65c | |||
| 8dce812a4d | |||
| 93047069b6 | |||
| c4f1990aff | |||
| 6e2816f08b | |||
| 227268024d | |||
| 8d5891a382 | |||
| 7700fca501 | |||
| 527c542d6d | |||
| 8fbae5e467 | |||
| 4d2a5efd12 | |||
| 5ffa14190a | |||
| 7820145cbe | |||
| b7a6c563ac | |||
| 52221488d0 | |||
| 4a1acb1446 | |||
| dc90211222 | |||
| 378c9f321d | |||
| e11bcc2848 | |||
| 3f10430150 | |||
| e8b72b54b3 | |||
| d902dda4b1 | |||
| 2538480b95 | |||
| b9b8c93cb9 | |||
| 68b7b35425 | |||
| 163c5feccc | |||
| 0488f0536e | |||
| 09e90ec25b | |||
| d97a11a54f | |||
| 4a779dfe3c | |||
| 61e09d545f | |||
| 3a68d7dabc | |||
| 22d318f201 | |||
| afa2cea678 | |||
| 6dce45505c | |||
| 014732788c | |||
| 0e75d838ab | |||
| 8383da8a50 | |||
| 199d173816 | |||
| 0e207dc5d2 |
@@ -0,0 +1,55 @@
|
||||
---
|
||||
description: Deploy the latest OmniRoute code to the Akamai VPS (69.164.221.35) via npm
|
||||
---
|
||||
|
||||
# Deploy to VPS Workflow
|
||||
|
||||
Deploy OmniRoute to the production VPS using Node.js + PM2 (no Docker).
|
||||
|
||||
**VPS:** `69.164.221.35` (Akamai, Ubuntu 24.04, 1GB RAM + 2.5GB swap)
|
||||
**App path:** `/opt/omniroute-app`
|
||||
**Process manager:** PM2 (`omniroute`)
|
||||
**Port:** `20128`
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Push to GitHub
|
||||
|
||||
Ensure all changes are committed and pushed:
|
||||
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
### 2. SSH into VPS, pull latest code, rebuild, and restart
|
||||
|
||||
// turbo-all
|
||||
|
||||
```bash
|
||||
ssh root@69.164.221.35 "
|
||||
cd /opt/omniroute-app &&
|
||||
git fetch origin &&
|
||||
git reset --hard origin/main &&
|
||||
export NODE_OPTIONS='--max-old-space-size=1536' &&
|
||||
npm install --no-audit --no-fund &&
|
||||
npm run build &&
|
||||
pm2 restart omniroute &&
|
||||
pm2 save &&
|
||||
echo '✅ Deploy complete!'
|
||||
"
|
||||
```
|
||||
|
||||
### 3. Verify the deployment
|
||||
|
||||
```bash
|
||||
ssh root@69.164.221.35 "pm2 list && curl -s -o /dev/null -w 'HTTP %{http_code}' http://localhost:20128/"
|
||||
```
|
||||
|
||||
Expected: PM2 shows `online`, HTTP returns `307` (redirect to login).
|
||||
|
||||
## Notes
|
||||
|
||||
- The VPS has only 1GB RAM. `NODE_OPTIONS='--max-old-space-size=1536'` uses swap for the build.
|
||||
- PM2 is configured with `pm2 startup` to auto-restart on reboot.
|
||||
- The `.env` file is at `/opt/omniroute-app/.env` (copied from the old Docker setup at `/opt/omniroute/.env`).
|
||||
- Nginx proxies `omniroute.online` → `localhost:20128`.
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
description: Create a new release, bump version up to 1.x.10 threshold, update changelog, and manage Pull Requests
|
||||
---
|
||||
|
||||
# Generate Release Workflow
|
||||
|
||||
Bump version, finalize CHANGELOG, commit, tag, push, publish to npm, and create GitHub release.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Determine new version
|
||||
|
||||
Check current version in `package.json` and increment the patch number:
|
||||
|
||||
```bash
|
||||
grep '"version"' package.json
|
||||
```
|
||||
|
||||
Version format: `1.x.y` — increment `y` for patch, `x` for minor (threshold: y=10 triggers x+1).
|
||||
|
||||
### 2. Finalize CHANGELOG.md
|
||||
|
||||
Replace `[Unreleased]` header with the new version and date:
|
||||
|
||||
```markdown
|
||||
## [1.x.y] — YYYY-MM-DD
|
||||
```
|
||||
|
||||
### 3. Bump version in package.json
|
||||
|
||||
```bash
|
||||
sed -i 's/"version": "OLD"/"version": "NEW"/' package.json
|
||||
```
|
||||
|
||||
### 4. Stage, commit, and tag
|
||||
|
||||
// turbo-all
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "feat(release): vX.Y.Z — summary of changes"
|
||||
git tag -a vX.Y.Z -m "Release vX.Y.Z — summary"
|
||||
```
|
||||
|
||||
### 5. Push to GitHub
|
||||
|
||||
```bash
|
||||
git push origin main
|
||||
git push origin vX.Y.Z
|
||||
```
|
||||
|
||||
### 6. Publish to npm
|
||||
|
||||
```bash
|
||||
npm publish
|
||||
```
|
||||
|
||||
Wait for completion (prepublishOnly runs `npm run build:cli` automatically).
|
||||
|
||||
### 7. Create GitHub release
|
||||
|
||||
```bash
|
||||
gh release create vX.Y.Z --title "Release vX.Y.Z" --notes-file /tmp/release_notes.md
|
||||
```
|
||||
|
||||
### 8. Deploy to VPS (if requested)
|
||||
|
||||
See `/deploy-vps` workflow for Akamai VPS or use npm for local VPS:
|
||||
|
||||
```bash
|
||||
ssh root@<VPS_IP> "npm install -g omniroute@X.Y.Z && pm2 restart omniroute"
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Always run `/update-docs` BEFORE this workflow (ensures CHANGELOG and README are current)
|
||||
- The `prepublishOnly` script runs `npm run build:cli` automatically during `npm publish`
|
||||
- After npm publish, verify with `npm info omniroute version`
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
description: How to respond to GitHub issues with insufficient information
|
||||
---
|
||||
|
||||
# Issue Triage Workflow
|
||||
|
||||
Respond to GitHub issues that need more information before they can be investigated.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Identify issues needing triage
|
||||
|
||||
```bash
|
||||
gh issue list --state open --limit 20
|
||||
```
|
||||
|
||||
### 2. Evaluate each issue
|
||||
|
||||
Check if the issue has:
|
||||
|
||||
- Clear reproduction steps
|
||||
- Environment details (OS, Node.js version, OmniRoute version)
|
||||
- Error logs/screenshots
|
||||
- Expected vs actual behavior
|
||||
|
||||
### 3. Respond with triage template
|
||||
|
||||
For issues missing information:
|
||||
|
||||
```markdown
|
||||
Thank you for reporting this issue! To help us investigate, please provide:
|
||||
|
||||
1. **OmniRoute version**: (`omniroute --version`)
|
||||
2. **Node.js version**: (`node --version`)
|
||||
3. **Operating system**: (e.g., Ubuntu 24.04, macOS 15, Windows 11)
|
||||
4. **Installation method**: (npm, Docker, source)
|
||||
5. **Steps to reproduce**: (exact commands/actions that trigger the issue)
|
||||
6. **Error logs**: (paste relevant logs from the console)
|
||||
7. **Expected behavior**: (what should happen)
|
||||
|
||||
This will help us debug and resolve your issue faster. 🙏
|
||||
```
|
||||
|
||||
### 4. Label the issue
|
||||
|
||||
Add appropriate labels: `needs-info`, `bug`, `enhancement`, `question`, etc.
|
||||
|
||||
```bash
|
||||
gh issue edit <NUMBER> --add-label "needs-info"
|
||||
```
|
||||
@@ -1,12 +1,12 @@
|
||||
---
|
||||
description: Fetch all open GitHub issues, analyze bugs, resolve what's possible, triage the rest, then commit and release
|
||||
description: Fetch all open GitHub issues, analyze bugs, resolve what's possible, triage the rest, wait for user validation, then commit and release
|
||||
---
|
||||
|
||||
# /resolve-issues — Automated Issue Resolution Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
This workflow fetches all open issues from the project's GitHub repository, classifies them, analyzes bugs, resolves what can be fixed, triages issues with insufficient information, and generates a release with all fixes.
|
||||
This workflow fetches all open issues from the project's GitHub repository, classifies them, analyzes bugs, resolves what can be fixed, and triages issues with insufficient information. **It does NOT merge or release automatically** — it creates a PR and waits for user validation before merging.
|
||||
|
||||
## Steps
|
||||
|
||||
@@ -60,41 +60,61 @@ Call the `/issue-triage` workflow (located at `~/.gemini/antigravity/global_work
|
||||
|
||||
Proceed with resolution:
|
||||
|
||||
1. **Research** — Search the codebase for files related to the issue
|
||||
2. **Root Cause** — Identify the root cause by reading the relevant source files
|
||||
3. **Implement Fix** — Apply the fix following existing code patterns and conventions
|
||||
4. **Test** — Build the project and run tests to verify the fix
|
||||
5. **Commit** — Commit with message format: `fix: <description> (#<issue_number>)`
|
||||
1. **Create a fix branch** — `git checkout -b fix/issue-<NUMBER>-<short-description>`
|
||||
2. **Research** — Search the codebase for files related to the issue
|
||||
3. **Root Cause** — Identify the root cause by reading the relevant source files
|
||||
4. **Implement Fix** — Apply the fix following existing code patterns and conventions
|
||||
5. **Test** — Build the project and run tests to verify the fix
|
||||
6. **Commit** — Commit with message format: `fix: <description> (#<issue_number>)`
|
||||
|
||||
### 5. Commit All Fixes
|
||||
### 5. Generate Report & Wait for Validation
|
||||
|
||||
After processing all issues:
|
||||
Present a summary report to the user via `notify_user` with `BlockedOnUser: true`:
|
||||
|
||||
- Ensure all fixes are committed with proper issue references
|
||||
- Each fix should be its own commit for clean git history
|
||||
| Issue | Title | Status | Action |
|
||||
| ----- | ----- | ------------- | ----------------------------- |
|
||||
| #N | Title | ✅ Ready | Files changed (not committed) |
|
||||
| #N | Title | ❓ Needs Info | Triage comment posted |
|
||||
| #N | Title | ⏭️ Skipped | Feature request / not a bug |
|
||||
|
||||
### 6. Close Resolved Issues
|
||||
> **⚠️ IMPORTANT**: Do NOT commit, close issues, or generate releases at this step.
|
||||
> Wait for the user to review the changes and respond with **OK** before proceeding.
|
||||
|
||||
For each successfully fixed issue:
|
||||
// turbo
|
||||
- If the user says **OK** or approves → Proceed to step 6
|
||||
- If the user requests changes → Apply the requested adjustments first, then present the report again
|
||||
- If the user rejects → Revert the changes and stop
|
||||
|
||||
- Close with a comment: `gh issue close <NUMBER> --repo <owner>/<repo> --comment "Fixed in <commit_hash>. The fix will be included in the next release."`
|
||||
### 6. Commit & Push Fix Branch (only after user approval)
|
||||
|
||||
### 7. Generate Report
|
||||
After the user validates:
|
||||
|
||||
Present a summary report to the user via `notify_user`:
|
||||
- Commit each fix individually with message format: `fix: <description> (#<issue_number>)`
|
||||
- Push the fix branch: `git push origin fix/issue-<NUMBER>-<short-description>`
|
||||
- Create a PR: `gh pr create --title "fix: <description> (#<issue_number>)" --body "<details>" --base main`
|
||||
|
||||
| Issue | Title | Status | Action |
|
||||
| ----- | ----- | ------------- | --------------------------- |
|
||||
| #N | Title | ✅ Fixed | Commit hash |
|
||||
| #N | Title | ❓ Needs Info | Triage comment posted |
|
||||
| #N | Title | ⏭️ Skipped | Feature request / not a bug |
|
||||
### 7. 🛑 WAIT — Notify User & Await PR Verification
|
||||
|
||||
### 8. Update Docs & Release
|
||||
**This is a mandatory stop point.** Use `notify_user` with `BlockedOnUser: true`:
|
||||
|
||||
If any fixes were committed:
|
||||
- Inform the user that the PR was created and is **awaiting their verification**
|
||||
- Include the PR number, URL, and a summary of what was changed
|
||||
- **DO NOT merge, close issues, generate releases, or deploy until the user confirms**
|
||||
|
||||
1. Run the `/update-docs` workflow (at `~/.gemini/antigravity/global_workflows/update-docs.md`) to update CHANGELOG and README
|
||||
2. Run the `/generate-release` workflow (at `.agents/workflows/generate-release.md`) to bump version, tag, and publish
|
||||
Wait for the user to respond:
|
||||
|
||||
- **User confirms** → Proceed to step 8
|
||||
- **User requests changes** → Apply changes, push to the same branch, notify again
|
||||
- **User rejects** → Close the PR and stop
|
||||
|
||||
### 8. Merge, Close Issues & Release (only after user confirms PR)
|
||||
|
||||
After the user confirms the PR:
|
||||
|
||||
1. **Merge** the PR: `gh pr merge <NUMBER> --merge --repo <owner>/<repo>` or via local merge
|
||||
2. **Close** resolved issues with a comment: `gh issue close <NUMBER> --repo <owner>/<repo> --comment "Fixed in <commit_hash>. The fix will be included in the next release."`
|
||||
3. **Switch to main**: `git checkout main && git pull`
|
||||
4. Run the `/update-docs` workflow (at `~/.gemini/antigravity/global_workflows/update-docs.md`) to update CHANGELOG and README
|
||||
5. Run the `/generate-release` workflow (at `.agents/workflows/generate-release.md`) to bump version, tag, and publish
|
||||
6. Deploy to local VPS: `ssh root@192.168.0.15 "npm install -g omniroute@<VERSION> && pm2 restart omniroute"`
|
||||
|
||||
If NO fixes were committed, skip this step and just present the report.
|
||||
|
||||
@@ -6,7 +6,7 @@ description: Analyze open Pull Requests from the project's GitHub repository, ge
|
||||
|
||||
## Overview
|
||||
|
||||
This workflow fetches all open PRs from the project's GitHub repository, performs a critical analysis of each one, generates a detailed report, and waits for user approval before proceeding with implementation.
|
||||
This workflow fetches all open PRs from the project's GitHub repository, performs a critical analysis of each one, generates a detailed report, and waits for user approval before proceeding with implementation. **All improvements are committed on top of the PR branch** and the user must verify before merge.
|
||||
|
||||
## Steps
|
||||
|
||||
@@ -60,6 +60,21 @@ This workflow fetches all open PRs from the project's GitHub repository, perform
|
||||
- Are edge cases covered?
|
||||
- Would existing tests break?
|
||||
|
||||
#### 3f. Cross-Layer (Global) Analysis
|
||||
|
||||
Perform a **global impact assessment** to verify whether the PR changes are complete across all layers of the application:
|
||||
|
||||
- **Backend → Frontend check**: If the PR adds or modifies backend-only resources (new endpoints, services, data models), evaluate whether corresponding frontend changes are missing:
|
||||
- Does a new endpoint require a new screen/page in the dashboard?
|
||||
- Should there be a new action button, menu item, or navigation link?
|
||||
- Are there new data fields that should be displayed or editable in the UI?
|
||||
- Does a new feature need a toggle, configuration panel, or status indicator?
|
||||
- **Frontend → Backend check**: If the PR adds frontend elements, verify the backend support exists:
|
||||
- Are the required API endpoints implemented?
|
||||
- Is the data model sufficient for the new UI components?
|
||||
- **Cross-cutting concerns**: Check shared layers (types, DTOs, validation schemas, routes, middleware) for completeness
|
||||
- **Document gaps** — If missing layers are detected, list them as **IMPORTANT** issues in the report with concrete suggestions for what should be added
|
||||
|
||||
### 4. Generate Report — Create a markdown report for each PR including:
|
||||
|
||||
- **PR Summary** — What it does, files affected, commit count
|
||||
@@ -79,18 +94,52 @@ This workflow fetches all open PRs from the project's GitHub repository, perform
|
||||
|
||||
### 6. Implementation (if approved)
|
||||
|
||||
- Checkout the PR branch or apply changes locally
|
||||
- Checkout the PR branch: `gh pr checkout <NUMBER>`
|
||||
- Implement any required fixes identified in the analysis
|
||||
- If the Cross-Layer Analysis (3f) identified missing frontend/backend counterparts, implement them
|
||||
- **Commit improvements on top of the PR branch** with descriptive commit messages
|
||||
- Run the project's test suite to verify nothing breaks
|
||||
// turbo
|
||||
- Run: `npm test` or equivalent test command
|
||||
- Build the project to verify compilation
|
||||
// turbo
|
||||
- Run: `npm run build` or equivalent build command
|
||||
- If all checks pass, prepare the merge
|
||||
- Push the updated branch: `git push origin <branch-name>`
|
||||
|
||||
### 7. Post-Merge (if applicable)
|
||||
### 7. 🛑 WAIT — Notify User & Await PR Verification
|
||||
|
||||
- Update CHANGELOG.md with the new feature
|
||||
- Consider version bump if warranted
|
||||
- Follow the `/generate-release` workflow if a release is needed
|
||||
**This is a mandatory stop point.** Use `notify_user` with `BlockedOnUser: true`:
|
||||
|
||||
- Inform the user that the PR has been **improved and pushed**, and is **awaiting their verification**
|
||||
- Include:
|
||||
- PR number and URL
|
||||
- Summary of improvements/fixes applied
|
||||
- Build/test status
|
||||
- List of files changed
|
||||
- **DO NOT merge, generate releases, or deploy until the user confirms**
|
||||
|
||||
Wait for the user to respond:
|
||||
|
||||
- **User confirms** → Proceed to step 8
|
||||
- **User requests more changes** → Apply changes, push to the same branch, notify again
|
||||
- **User rejects** → Leave a review comment and stop
|
||||
|
||||
### 8. Thank the Contributor
|
||||
|
||||
- Post a **thank-you comment** on the PR via the GitHub API
|
||||
- The message should:
|
||||
- Thank the author by name/username for their contribution
|
||||
- Briefly mention what the PR accomplishes and any improvements applied
|
||||
- Be friendly, professional, and encouraging
|
||||
- Example: _"Thanks @author for this great contribution! 🎉 The [feature/fix] is now merged and will be part of the next release. We appreciate your effort!"_
|
||||
|
||||
### 9. Merge & Release (only after user confirms PR)
|
||||
|
||||
After the user confirms the PR:
|
||||
|
||||
1. **Merge** the PR into main (local merge with `--no-ff` or via `gh pr merge`)
|
||||
2. **Push** to main: `git push origin main`
|
||||
3. **Clean up** the feature branch: `git branch -d <branch-name>`
|
||||
4. **Update CHANGELOG.md** with the new feature/fix
|
||||
5. Run the `/generate-release` workflow (at `.agents/workflows/generate-release.md`) to bump version, tag, and publish
|
||||
6. Deploy to local VPS: `ssh root@192.168.0.15 "npm install -g omniroute@<VERSION> && pm2 restart omniroute"`
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
description: How to automatically summarize recent changes and update README and CHANGELOG
|
||||
---
|
||||
|
||||
# Update Documentation Workflow
|
||||
|
||||
Update CHANGELOG.md, README.md, docs/ files, and all multi-language translations whenever features are added or changed.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Summarize recent changes
|
||||
|
||||
Review git log and identify new features, fixes, or changes since the last release tag:
|
||||
|
||||
```bash
|
||||
git log $(git describe --tags --abbrev=0)..HEAD --oneline
|
||||
```
|
||||
|
||||
### 2. Update English CHANGELOG.md
|
||||
|
||||
Add an `[Unreleased]` section (or version header if releasing) with:
|
||||
|
||||
- `### ✨ New Features` — each feature as a bullet point
|
||||
- `### 🐛 Bug Fixes` — if applicable
|
||||
- `### 🧪 Tests` — test count changes
|
||||
- `### 📁 New Files` — table of new files with purpose
|
||||
|
||||
### 3. Update English README.md
|
||||
|
||||
Update the feature tables in these sections:
|
||||
|
||||
- **🧠 Routing & Intelligence** — for routing/model features
|
||||
- **🛡️ Resilience & Security** — for security/resilience features
|
||||
- **📊 Observability & Analytics** — for monitoring features
|
||||
- **☁️ Deploy & Sync** — for deployment features
|
||||
|
||||
### 4. Update docs/ files
|
||||
|
||||
- `docs/FEATURES.md` — update the Settings section description
|
||||
- `docs/API_REFERENCE.md` — add new API routes if any
|
||||
- `docs/ARCHITECTURE.md` — update architecture if structural changes
|
||||
|
||||
### 5. 🌐 Sync Multi-Language Documentation (CRITICAL)
|
||||
|
||||
// turbo-all
|
||||
|
||||
**This step MUST be run after every README or docs update.**
|
||||
|
||||
The project has **30 language versions** of documentation:
|
||||
|
||||
**README files (root directory):**
|
||||
|
||||
```
|
||||
README.md (English - source of truth)
|
||||
README.pt-BR.md README.pt.md README.es.md README.fr.md README.it.md
|
||||
README.de.md README.nl.md README.sv.md README.no.md README.da.md README.fi.md
|
||||
README.ru.md README.uk-UA.md README.bg.md README.sk.md README.pl.md README.ro.md README.hu.md
|
||||
README.ar.md README.he.md README.th.md README.in.md README.id.md README.ms.md README.vi.md
|
||||
README.ja.md README.ko.md README.zh-CN.md README.phi.md
|
||||
```
|
||||
|
||||
**docs/i18n/ directories (29 languages):**
|
||||
|
||||
```
|
||||
docs/i18n/{ar,bg,da,de,es,fi,fr,he,hu,id,in,it,ja,ko,ms,nl,no,phi,pl,pt,pt-BR,ro,ru,sk,sv,th,uk-UA,vi,zh-CN}/
|
||||
Each contains: API_REFERENCE.md, ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, FEATURES.md, TROUBLESHOOTING.md, USER_GUIDE.md
|
||||
```
|
||||
|
||||
**Sync approach for feature table updates:**
|
||||
|
||||
a. Identify which feature table rows were added to English README.md
|
||||
b. For each translated README, find the corresponding anchor lines:
|
||||
|
||||
- **Routing section:** Find the `💬` (System Prompt) table row — the line before it is always the last routing feature. Insert new routing features before System Prompt.
|
||||
- **Resilience section:** Find the `📊` Rate Limits table row (the one in lines 590-600, NOT the quota tracking one in lines 560-570). Insert new resilience features after it.
|
||||
c. The new feature entries can stay in English for technical features, matching the pattern used in the existing translations.
|
||||
d. Use `sed` or similar tool to batch-insert across all 29 translated READMEs.
|
||||
|
||||
**Verification:**
|
||||
|
||||
```bash
|
||||
# Verify all READMEs have the new features
|
||||
grep -l "NEW_FEATURE_NAME" README.*.md | wc -l
|
||||
# Should return 30 (all language versions)
|
||||
```
|
||||
|
||||
**FEATURES.md sync:**
|
||||
|
||||
```bash
|
||||
# Update Settings description in all docs/i18n/*/FEATURES.md
|
||||
for dir in docs/i18n/*/; do
|
||||
# Update the Settings section description to mention new features
|
||||
# Check FEATURES.md in each directory
|
||||
done
|
||||
```
|
||||
|
||||
### 6. Verify documentation changes
|
||||
|
||||
```bash
|
||||
# Check all modified files
|
||||
git status --short
|
||||
|
||||
# Verify no broken markdown
|
||||
# Optional: run markdownlint if available
|
||||
```
|
||||
@@ -130,6 +130,22 @@ GEMINI_CLI_OAUTH_CLIENT_SECRET=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
|
||||
# IFLOW_OAUTH_CLIENT_ID=
|
||||
IFLOW_OAUTH_CLIENT_SECRET=4Z3YjXycVsQvyGF1etiNlIBB4RsqSDtW
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Provider User-Agent Overrides (optional — customize per-provider UA headers)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Format: {PROVIDER_ID}_USER_AGENT=custom-value
|
||||
# When set, overrides the default User-Agent header sent to that provider.
|
||||
# Useful when providers update versions or block old user-agents.
|
||||
CLAUDE_USER_AGENT=claude-cli/1.0.83 (external, cli)
|
||||
CODEX_USER_AGENT=codex-cli/0.92.0 (Windows 10.0.26100; x64)
|
||||
GITHUB_USER_AGENT=GitHubCopilotChat/0.26.7
|
||||
ANTIGRAVITY_USER_AGENT=antigravity/1.104.0 darwin/arm64
|
||||
KIRO_USER_AGENT=AWS-SDK-JS/3.0.0 kiro-ide/1.0.0
|
||||
IFLOW_USER_AGENT=iFlow-Cli
|
||||
QWEN_USER_AGENT=google-api-nodejs-client/9.15.1
|
||||
CURSOR_USER_AGENT=connect-es/1.6.1
|
||||
GEMINI_CLI_USER_AGENT=google-api-nodejs-client/9.15.1
|
||||
|
||||
# API Key Providers (Phase 1 + Phase 4)
|
||||
# Add via Dashboard → Providers → Add API Key, or set here
|
||||
# DEEPSEEK_API_KEY=
|
||||
|
||||
@@ -3,11 +3,11 @@ name: Build Electron Desktop App
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Release version (e.g., v1.6.8)'
|
||||
description: "Release version (e.g., v1.6.8)"
|
||||
required: true
|
||||
type: string
|
||||
|
||||
@@ -89,6 +89,8 @@ jobs:
|
||||
run: npm ci
|
||||
|
||||
- name: Build Next.js standalone
|
||||
env:
|
||||
JWT_SECRET: ci-build-secret-with-sufficient-length-for-validation
|
||||
run: npm run build
|
||||
|
||||
- name: Install Electron dependencies
|
||||
@@ -100,6 +102,7 @@ jobs:
|
||||
run: npm run build:${{ matrix.target }}
|
||||
|
||||
- name: Collect installers
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p release-assets
|
||||
cd electron/dist-electron
|
||||
@@ -139,11 +142,11 @@ jobs:
|
||||
# Create source code archives (excluding dev dependencies and build artifacts)
|
||||
export TARBALL="OmniRoute-${{ needs.validate.outputs.version }}.source.tar.gz"
|
||||
export ZIPBALL="OmniRoute-${{ needs.validate.outputs.version }}.source.zip"
|
||||
|
||||
|
||||
# Use git archive for clean source export
|
||||
git archive --format=tar.gz --prefix=OmniRoute-${{ needs.validate.outputs.version }}/ HEAD -o "release-assets/$TARBALL"
|
||||
git archive --format=zip --prefix=OmniRoute-${{ needs.validate.outputs.version }}/ HEAD -o "release-assets/$ZIPBALL"
|
||||
|
||||
|
||||
echo "✓ Created source archives:"
|
||||
ls -lh "release-assets/$TARBALL" "release-assets/$ZIPBALL"
|
||||
|
||||
|
||||
+179
@@ -7,6 +7,185 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
---
|
||||
|
||||
## [1.7.11] — 2026-03-02
|
||||
|
||||
### ✨ Features
|
||||
|
||||
- **Blackbox AI Provider** — Added blackbox.ai as a new OpenAI-compatible provider with 6 default models (GPT-4o, Gemini 2.5 Flash, Claude Sonnet 4, DeepSeek V3, Blackbox AI, Blackbox AI Pro) and provider logo. Fixes #175
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **Antigravity 404 Error** — Added warning logs when `generateProjectId()` generates a fallback project ID because `credentials.projectId` is null. The executor now prefers the translator-set `body.project` before generating a new fallback, eliminating duplicate warnings and ID mismatch. Fixes #176. Includes improvements from PRs #184 and #185
|
||||
|
||||
## [1.7.10] — 2026-03-02
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **Streaming Tool Calls (Responses→ChatCompletions)** — Fixed two issues in the `openaiResponsesToOpenAIResponse` translator that broke tool call execution in agentic clients (OpenCode, Claude Code, Cursor, etc.): (1) Argument delta chunks now include `tool_calls[].id` and `type: "function"` so clients can associate argument fragments correctly. (2) `finish_reason` is now `"tool_calls"` instead of hardcoded `"stop"` when tool calls occurred. Fixes #180
|
||||
|
||||
## [1.7.9] — 2026-03-02
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **Electron CI Build** — Added `JWT_SECRET` environment variable to the Electron release workflow `Build Next.js standalone` step, fixing build failures in GitHub Actions. PR #178 by @benzntech
|
||||
|
||||
### 📝 Documentation
|
||||
|
||||
- **README** — Updated OpenClaw link from `cline/cline` to `openclaw/openclaw` to reflect the project rename. PR #179 by @MAINER4IK
|
||||
|
||||
## [1.7.8] — 2026-03-02
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **Theme Color Customization** — Users can now select from 7 preset accent colors (Coral, Blue, Red, Green, Violet, Orange, Cyan) or define a custom color via color picker/hex input. The chosen color dynamically updates `--color-primary` and `--color-primary-hover` CSS variables across the entire UI. PR #174 by @mainer4ik
|
||||
|
||||
### 🌐 Multi-Language Sync
|
||||
|
||||
- **Theme & Media i18n** — Added `themeCoral`, `themeBlue`, `themeRed`, `themeGreen`, `themeViolet`, `themeOrange`, `themeCyan`, `themeAccent`, `themeAccentDesc`, `themeCustom`, `themeCreate`, and media section translations across all **30 language locales**
|
||||
|
||||
### 🔧 Code Quality (Review Improvements)
|
||||
|
||||
- Exported `COLOR_THEMES` constant from `themeStore.ts` for DRY reuse
|
||||
- Added hex color validation with visual feedback (red border + disabled apply button)
|
||||
- Synced local state via Zustand `subscribe` pattern for cross-tab consistency
|
||||
- Removed dead `/themes` route from Header.tsx
|
||||
- Added CSS `color-mix()` fallback for older browsers
|
||||
|
||||
## [1.7.7] — 2026-03-02
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **Gemini Tool Schema Sanitization** — The standard Gemini provider now sanitizes OpenAI tool schemas before forwarding to Gemini API, removing unsupported JSON Schema keywords (`additionalProperties`, `$schema`, `const`, `default`, `not`, etc.). Previously, sanitization only ran in the CLI executor path, causing Gemini to reject tool calls when schemas contained unsupported constraints. Also applied sanitization to `response_format.json_schema`. Fixes #173
|
||||
|
||||
## [1.7.6] — 2026-03-02
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **Cloud Proxy `undefined/v1` Fix** — When the `NEXT_PUBLIC_CLOUD_URL` environment variable is not set (common in Docker deployments), the endpoint page now correctly falls back instead of showing `undefined/v1`. The cloud sync API now returns `cloudUrl` in its response so the frontend can use it dynamically. Fixes #171
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **Cloud Worker `/v1/models` Endpoint** — The Cloud Worker now supports the `/v1/models` endpoint for both URL formats (`/v1/models` and `/{machineId}/v1/models`), returning all available models synced from the local OmniRoute instance
|
||||
|
||||
### 🔧 Infrastructure
|
||||
|
||||
- **Cloudflare Workers Compatibility** — Fixed `setInterval` in global scope issue in `accountFallback.ts` that blocked Cloud Worker deployment. Lazy initialization pattern ensures compatibility with Cloudflare Workers runtime restrictions
|
||||
|
||||
## [1.7.5] — 2026-03-02
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **OAuth Re-Auth Duplicate Fix** — Re-authenticating an expired OAuth connection now updates the existing connection instead of creating a duplicate entry. When re-auth is triggered, the system matches by `provider` + `email` + `authType` and refreshes tokens in-place. Fixes #170
|
||||
|
||||
## [1.7.4] — 2026-03-01
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **OpenCode CLI Integration** — Added full integration guide for [OpenCode](https://opencode.ai) AI CLI tool using `@ai-sdk/openai-compatible` adapter with custom `opencode.json` config. Resolves #169
|
||||
- **Endpoint Page Restructured** — Reorganized the Endpoint dashboard page into 3 grouped categories (Core APIs, Media & Multi-Modal, Utility & Management) with visual dividers. Added 2 new endpoint sections: **Responses API** (`/v1/responses`) and **List Models** (`/v1/models`)
|
||||
- **Model Aliases & Background Degradation i18n** — Added 14 translated settings keys and 7 translated endpoint keys across all **30 language locales**. Fixed missing translations showing raw keys like `settings.modelAliasesTitle` in the UI
|
||||
|
||||
### 🌐 Multi-Language Sync
|
||||
|
||||
- **30 README translations synced** — All 28 translated READMEs updated with v1.7.3 feature entries (Model Aliases, Background Degradation, Rate Limit Persistence, Token Refresh Resilience)
|
||||
- **6 docs/i18n FEATURES.md updated** — Settings description expanded in da, it, nl, phi, pl, sv
|
||||
|
||||
### 📁 New Files
|
||||
|
||||
| File | Purpose |
|
||||
| --------------------------------------- | ----------------------------------------------------------- |
|
||||
| `.agents/workflows/update-docs.md` | Documentation update workflow with multi-language sync step |
|
||||
| `.agents/workflows/generate-release.md` | Release generation workflow (version bump, npm, GitHub) |
|
||||
| `.agents/workflows/issue-triage.md` | Issue triage workflow for issues with insufficient info |
|
||||
|
||||
## [1.7.3] — 2026-03-01
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **Model Deprecation Auto-Forward** — New `modelDeprecation.ts` service with 10+ built-in aliases for legacy Gemini, Claude, and OpenAI models. Deprecated model IDs (e.g., `gemini-pro`, `claude-2`) are automatically forwarded to their current replacements. Custom aliases configurable via new Settings → Routing → Model Aliases UI tab with full CRUD API (`/api/settings/model-aliases`)
|
||||
- **Background Task Smart Degradation** — New `backgroundTaskDetector.ts` service detects background/utility requests (title generation, summarization, etc.) via 19 system prompt patterns and `X-Request-Priority` header, and automatically reroutes them to cheaper models. Configurable degradation map and detection patterns via new Settings → Routing → Background Degradation UI tab. Disabled by default (opt-in)
|
||||
- **Rate Limit Persistence** — Learned rate limits from API response headers are now persisted to SQLite with 60-second debouncing and restored on startup (24h staleness filter). Rate limits survive server restarts instead of being lost in memory
|
||||
- **thinkingLevel String Conversion** — `applyThinkingBudget()` now handles string-based `thinkingLevel` inputs (`"high"`, `"medium"`, `"low"`, `"none"`) by converting them to numeric token budgets. Supports `thinkingLevel`, `thinking_level`, and Gemini's `generationConfig.thinkingConfig.thinkingLevel` fields
|
||||
- **Claude -thinking Model Auto-Injection** — Models ending with `-thinking` suffix (e.g., `claude-opus-4-6-thinking`) automatically get thinking parameters injected to prevent API errors. `hasThinkingCapableModel()` updated to recognize these suffixes
|
||||
- **Gemini 3.0/3.1 Model Registry** — Updated provider registry to explicitly distinguish Gemini 3.1 (Pro, Flash) from 3.0 Preview variants across `gemini`, `gemini-cli`, and `antigravity` providers with clear naming conventions
|
||||
- **Token Refresh Circuit Breaker** — Per-provider circuit breaker in `refreshWithRetry()`: 5 consecutive failures trigger a 30-minute cooldown to prevent infinite retry loops. Added 30-second timeout wrapper per refresh attempt. Exported `isProviderBlocked()` and `getCircuitBreakerStatus()` for diagnostics
|
||||
|
||||
### 🧪 Tests
|
||||
|
||||
- **40+ new unit tests** across 3 files: `model-deprecation.test.mjs` (14 tests), `background-task-detector.test.mjs` (14 tests), extended `thinking-budget.test.mjs` (+13 tests). Total suite: **561 tests, 0 failures**
|
||||
|
||||
### 📁 New Files
|
||||
|
||||
| File | Purpose |
|
||||
| ---------------------------------------------------------------------- | --------------------------------------------------------------------- |
|
||||
| `open-sse/services/modelDeprecation.ts` | Model deprecation alias resolver with built-in + custom aliases |
|
||||
| `open-sse/services/backgroundTaskDetector.ts` | Background task detection with pattern matching and model degradation |
|
||||
| `src/app/api/settings/model-aliases/route.ts` | CRUD API for model alias management |
|
||||
| `src/app/api/settings/background-degradation/route.ts` | API for background degradation config |
|
||||
| `src/app/(dashboard)/settings/components/ModelAliasesTab.tsx` | Settings UI for model alias management |
|
||||
| `src/app/(dashboard)/settings/components/BackgroundDegradationTab.tsx` | Settings UI for background degradation |
|
||||
| `tests/unit/model-deprecation.test.mjs` | 14 unit tests for model deprecation |
|
||||
| `tests/unit/background-task-detector.test.mjs` | 14 unit tests for background task detection |
|
||||
|
||||
---
|
||||
|
||||
## [1.7.2] — 2026-03-01
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **Multi-Modal Provider Support** — Added 6 TTS providers (ElevenLabs, Nvidia NIM, HuggingFace, Coqui, Tortoise, Qwen3), 3 STT providers, 2 image providers (SD WebUI, ComfyUI), and two new modalities: `/v1/videos/generations` (Text-to-Video) and `/v1/music/generations` (Text-to-Music). Shared abstractions via `registryUtils.ts` and `comfyuiClient.ts` ([PR #167](https://github.com/diegosouzapw/OmniRoute/pull/167) by @ken2190)
|
||||
- **Media Playground Page** — New dashboard page at `/dashboard/media` with tabbed interface (Image/Video/Music), model selector, prompt input, and JSON result viewer
|
||||
- **Unit Tests for Registry Utils** — 24 tests covering `parseModelFromRegistry`, `getAllModelsFromRegistry`, `buildAuthHeaders`, and integration with video/music registries
|
||||
- **WFGY 16-Problem RAG Failure Map** — Added troubleshooting reference for RAG/LLM failure taxonomy in `docs/TROUBLESHOOTING.md` ([PR #164](https://github.com/diegosouzapw/OmniRoute/pull/164) by @onestardao)
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Gemini Imported Models Return 404** — Strip `models/` prefix from Gemini model IDs during import to prevent doubled paths ([#163](https://github.com/diegosouzapw/OmniRoute/issues/163))
|
||||
- **Pino File Transport Fails in Next.js Production** — Log actual error + add sync `pino.destination()` fallback ([#165](https://github.com/diegosouzapw/OmniRoute/issues/165))
|
||||
- **Windows Electron CI Build** — Added `shell: bash` to Collect installers step for Windows runners ([PR #168](https://github.com/diegosouzapw/OmniRoute/pull/168) by @benzntech)
|
||||
- **TypeScript Safety** — Replaced `Record<string, any>` with `Record<string, unknown>` in `registryUtils.ts`
|
||||
|
||||
---
|
||||
|
||||
## [1.7.1] — 2026-02-28
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Dashboard Layout Breakage** — Tailwind CSS v4 auto-detection failed to scan Next.js route group directories with parentheses (e.g. `(dashboard)`), causing all responsive grid utilities (`sm:grid-cols-*`, `md:grid-cols-*`, `lg:grid-cols-*`, `xl:grid-cols-*`) to be purged from production CSS. Cards displayed in a single column instead of multi-column grids. Fixed by adding explicit `@source` directives in `globals.css`
|
||||
|
||||
---
|
||||
|
||||
## [1.7.0] — 2026-02-28
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **16 Pain Points Documentation** — New collapsible section "🎯 What OmniRoute Solves — 16 Real Pain Points" added to the main README and all 29 language-specific READMEs. Each pain point uses `<details>/<summary>` tags for clean, expandable content
|
||||
- **Configurable User-Agent per Provider** — User-Agent strings for OAuth providers (Claude, Codex, GitHub, Antigravity, Kiro, iFlow, Qwen, Cursor, Gemini CLI) are now configurable via environment variables. Format: `{PROVIDER_ID}_USER_AGENT=custom-value` ([#155](https://github.com/diegosouzapw/OmniRoute/issues/155))
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Hardcoded `$HOME` Path in Standalone/Bun Builds** — 5 files (`backupService.ts`, `mitm/manager.ts`, `mitm/server.ts`, `mitm/cert/generate.ts`, `codex-profiles/route.ts`) were bypassing the centralized `dataPaths.ts` and using `os.homedir()` directly. This caused paths to bake the build machine's `$HOME` into standalone/bun builds, producing `EACCES: permission denied` errors on other machines. All files now use `resolveDataDir()` from `dataPaths.ts`, respecting `DATA_DIR` env var and XDG conventions ([#156](https://github.com/diegosouzapw/OmniRoute/issues/156))
|
||||
|
||||
### 📝 Documentation
|
||||
|
||||
- **`.env` and `.env.example` Synced** — Added 9 User-Agent env vars with latest known default values to both environment files
|
||||
- **30 README Translations Updated** — All language READMEs now include the 16 Pain Points section
|
||||
|
||||
---
|
||||
|
||||
## [1.6.9] — 2026-02-28
|
||||
|
||||
### 🐛 Fixed
|
||||
|
||||
- **Proxy Port Preservation** — `new URL()` silently strips default ports (80/443); proxy connections now extract the port from the raw URL string before parsing, preventing connection timeouts ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **Proxy Credential Encoding** — URL-encode special characters in proxy username/password; decode during legacy migration ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **HTTPS Proxy Default Port** — Changed from 8080 to 443 in frontend and migration logic ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **Proxy Dispatcher Cache** — Invalidate cached dispatchers when proxy config is updated or deleted ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **Proxy Logger SQLite Type** — Cast `proxyPort` to `Number` for INTEGER column ([PR #161](https://github.com/diegosouzapw/OmniRoute/pull/161))
|
||||
- **CopilotToolCard URL** — Use `baseUrl` prop directly instead of redundant `window.location.origin`; filter to chat models only (`!m.type && !m.parent`) ([PR #160](https://github.com/diegosouzapw/OmniRoute/pull/160))
|
||||
|
||||
---
|
||||
|
||||
## [1.6.8] — 2026-02-28
|
||||
|
||||
### 🔧 Improved
|
||||
|
||||
@@ -2,6 +2,7 @@ FROM node:22-bookworm-slim AS builder
|
||||
WORKDIR /app
|
||||
|
||||
COPY package*.json ./
|
||||
COPY scripts/postinstall.mjs ./scripts/postinstall.mjs
|
||||
RUN if [ -f package-lock.json ]; then npm ci --no-audit --no-fund; else npm install --no-audit --no-fund; fi
|
||||
|
||||
COPY . ./
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ بداية سريعة
|
||||
|
||||
**1. التثبيت عالميًا:**
|
||||
@@ -310,6 +567,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **نماذج مخصصة** | أضف أي معرف نموذج إلى أي مزود |
|
||||
| 🌐 **جهاز توجيه Wildcard** | قم بتوجيه أنماط `provider/*` إلى أي مزود ديناميكيًا |
|
||||
| 🧠 **ميزانية التفكير** | أوضاع العبور والتلقائي والمخصص والتكيفي لنماذج الاستدلال |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **الحقن الفوري للنظام** | يتم تطبيق موجه النظام العالمي على كافة الطلبات |
|
||||
| 📄 **Responses API** | دعم واجهة برمجة تطبيقات استجابات OpenAI الكاملة (`/v1/responses`) لـ Codex |
|
||||
|
||||
@@ -335,6 +594,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **انتحال بصمة الإصبع TLS** | تجاوز اكتشاف الروبوتات المستندة إلى TLS عبر wreq-js |
|
||||
| 🌐 **تصفية IP** | القائمة المسموح بها/القائمة المحظورة للتحكم في الوصول إلى واجهة برمجة التطبيقات |
|
||||
| 📊 **حدود المعدل القابلة للتحرير** | عدد الدورات في الدقيقة القابل للتكوين والفجوة الدنيا والحد الأقصى المتزامن على مستوى النظام |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **حماية نقطة نهاية واجهة برمجة التطبيقات** | بوابة المصادقة + حظر الموفر لنقطة النهاية `/models` |
|
||||
| 🔒 **رؤية الوكيل** | شارات مرمزة بالألوان: 🟢 عالمية، 🟡 مزود، 🔵 لكل اتصال مع عرض IP |
|
||||
| 🌐 ** تكوين الوكيل ذو 3 مستويات ** | قم بتكوين الوكلاء على المستوى العالمي أو لكل مزود أو لكل اتصال |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Бърз старт
|
||||
|
||||
**1. Инсталирайте глобално:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Персонализирани модели** | Добавете всеки модел ID към който и да е доставчик |
|
||||
| 🌐 **Wildcard Router** | Насочвайте `provider/*` шаблони към всеки доставчик динамично |
|
||||
| 🧠 **Мислен бюджет** | Преминаване, автоматичен, персонализиран и адаптивен режим за модели на разсъждение |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Системно бързо инжектиране** | Глобална системна подкана, приложена към всички заявки |
|
||||
| 📄 **API за отговори** | Пълна поддръжка на OpenAI Responses API (`/v1/responses`) за Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Заобикаляне на TLS-базирано откриване на бот чрез wreq-js |
|
||||
| 🌐 **IP филтриране** | Списък с разрешени/блокирани за контрол на достъпа до API |
|
||||
| 📊 **Редактируеми ограничения на скоростта** | Конфигурируеми обороти в минута, минимална разлика и максимална едновременност на системно ниво |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Удостоверяване + блокиране на доставчик за крайната точка `/models` |
|
||||
| 🔒 **Прокси видимост** | Цветно кодирани значки: 🟢 глобален, 🟡 доставчик, 🔵 за връзка с IP дисплей |
|
||||
| 🌐 **3-ниво на прокси конфигурация** | Конфигуриране на прокси сървъри на глобално ниво, на ниво доставчик или на ниво връзка |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Hurtig start
|
||||
|
||||
**1. Installer globalt:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Tilpassede modeller** | Tilføj ethvert model-id til enhver udbyder |
|
||||
| 🌐 **Wildcard-router** | Rut `provider/*` mønstre til enhver udbyder dynamisk |
|
||||
| 🧠 **Tænkende budget** | Passthrough, auto, brugerdefinerede og adaptive tilstande til ræsonnerende modeller |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Injection** | Global systemprompt anvendt på tværs af alle anmodninger |
|
||||
| 📄 **Responses API** | Fuld OpenAI Responses API (`/v1/responses`) understøttelse af Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Omgå TLS-baseret botdetektion via wreq-js |
|
||||
| 🌐 **IP-filtrering** | Tilladelsesliste/blokeringsliste til API-adgangskontrol |
|
||||
| 📊 **Redigerbare satsgrænser** | Konfigurerbar RPM, min. gap og maks. samtidighed på systemniveau |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API-endepunktsbeskyttelse** | Auth gating + udbyderblokering for `/models` slutpunktet |
|
||||
| 🔒 **Proxysynlighed** | Farvekodede badges: 🟢 global, 🟡 udbyder, 🔵 per forbindelse med IP-skærm |
|
||||
| 🌐 **3-Level Proxy Config** | Konfigurer proxyer på globalt niveau, pr. udbyder eller pr. forbindelsesniveau |
|
||||
|
||||
+270
-9
@@ -157,6 +157,263 @@ Ergebnis: Nie aufhören zu programmieren, minimale Kosten
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Schnellstart
|
||||
|
||||
**1. Global installieren:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Benutzerdefinierte Modelle** | Jede Modell-ID zu jedem Anbieter hinzufügen |
|
||||
| 🌐 **Wildcard-Router** | `provider/*` Muster dynamisch an jeden Anbieter routen |
|
||||
| 🧠 **Reasoning-Budget** | Passthrough, auto, custom und adaptive Modi für Reasoning-Modelle |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Injection** | Globaler System Prompt für alle Anfragen |
|
||||
| 📄 **API Responses** | Volle Unterstützung der OpenAI Responses API (`/v1/responses`) für Codex |
|
||||
|
||||
@@ -327,15 +586,17 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ Resilienz & Sicherheit
|
||||
|
||||
| Funktion | Was es macht |
|
||||
| ------------------------------- | -------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Auto-Öffnung/-Schließung pro Anbieter mit konfigurierbaren Schwellen |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + Semaphor Rate-Limit für API-Key-Anbieter |
|
||||
| 🧠 **Semantischer Cache** | Zwei-Ebenen-Cache (Signatur + Semantik) senkt Kosten und Latenz |
|
||||
| ⚡ **Anfrage-Idempotenz** | 5s Dedup-Fenster für doppelte Anfragen |
|
||||
| 🔒 **TLS-Fingerprint-Spoofing** | Bot-Erkennung umgehen via wreq-js |
|
||||
| 🌐 **IP-Filterung** | Allowlist/Blocklist für API-Zugriffskontrolle |
|
||||
| 📊 **Editierbare Rate-Limits** | Konfigurierbare RPM, minimaler Abstand, max. Konkurrenz |
|
||||
| Funktion | Was es macht |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Auto-Öffnung/-Schließung pro Anbieter mit konfigurierbaren Schwellen |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + Semaphor Rate-Limit für API-Key-Anbieter |
|
||||
| 🧠 **Semantischer Cache** | Zwei-Ebenen-Cache (Signatur + Semantik) senkt Kosten und Latenz |
|
||||
| ⚡ **Anfrage-Idempotenz** | 5s Dedup-Fenster für doppelte Anfragen |
|
||||
| 🔒 **TLS-Fingerprint-Spoofing** | Bot-Erkennung umgehen via wreq-js |
|
||||
| 🌐 **IP-Filterung** | Allowlist/Blocklist für API-Zugriffskontrolle |
|
||||
| 📊 **Editierbare Rate-Limits** | Konfigurierbare RPM, minimaler Abstand, max. Konkurrenz |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
|
||||
### 📊 Observability & Analytics
|
||||
|
||||
|
||||
+270
-9
@@ -157,6 +157,263 @@ Resultado: Nunca dejes de programar, costo mínimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Inicio Rápido
|
||||
|
||||
**1. Instala globalmente:**
|
||||
@@ -315,6 +572,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Modelos Personalizados** | Agrega cualquier ID de modelo a cualquier proveedor |
|
||||
| 🌐 **Enrutador Wildcard** | Enruta patrones `provider/*` a cualquier proveedor dinámicamente |
|
||||
| 🧠 **Presupuesto de Razonamiento** | Modos passthrough, auto, custom y adaptativo para modelos de razonamiento |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Inyección de System Prompt** | System prompt global aplicado en todas las solicitudes |
|
||||
| 📄 **API Responses** | Soporte completo de la API Responses de OpenAI (`/v1/responses`) para Codex |
|
||||
|
||||
@@ -331,15 +590,17 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ Resiliencia y Seguridad
|
||||
|
||||
| Característica | Qué Hace |
|
||||
| ---------------------------------- | ---------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Auto-apertura/cierre por proveedor con umbrales configurables |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semáforo rate-limit para proveedores con API key |
|
||||
| 🧠 **Caché Semántico** | Caché de dos niveles (firma + semántico) reduce costo y latencia |
|
||||
| ⚡ **Idempotencia de Solicitud** | Ventana de dedup de 5s para solicitudes duplicadas |
|
||||
| 🔒 **Spoofing de Fingerprint TLS** | Bypass de detección de bot vía TLS con wreq-js |
|
||||
| 🌐 **Filtrado de IP** | Allowlist/blocklist para control de acceso a la API |
|
||||
| 📊 **Rate Limits Editables** | RPM, gap mínimo y concurrencia máxima configurables |
|
||||
| Característica | Qué Hace |
|
||||
| ---------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Auto-apertura/cierre por proveedor con umbrales configurables |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semáforo rate-limit para proveedores con API key |
|
||||
| 🧠 **Caché Semántico** | Caché de dos niveles (firma + semántico) reduce costo y latencia |
|
||||
| ⚡ **Idempotencia de Solicitud** | Ventana de dedup de 5s para solicitudes duplicadas |
|
||||
| 🔒 **Spoofing de Fingerprint TLS** | Bypass de detección de bot vía TLS con wreq-js |
|
||||
| 🌐 **Filtrado de IP** | Allowlist/blocklist para control de acceso a la API |
|
||||
| 📊 **Rate Limits Editables** | RPM, gap mínimo y concurrencia máxima configurables |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
|
||||
### 📊 Observabilidad y Analytics
|
||||
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Pika-aloitus
|
||||
|
||||
**1. Asenna maailmanlaajuisesti:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Räätälöidyt mallit** | Lisää mikä tahansa mallitunnus mille tahansa toimittajalle |
|
||||
| 🌐 **Wildcard-reititin** | Reititä `provider/*` mallit mille tahansa palveluntarjoajalle dynaamisesti |
|
||||
| 🧠 **Ajatteleva budjetti** | Läpivienti-, automaatti-, mukautetut ja mukautuvat tilat päättelymalleille |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Järjestelmän pikaruiskutus** | Maailmanlaajuinen järjestelmäkehote käytössä kaikissa pyynnöissä |
|
||||
| 📄 **Responses API** | Täysi OpenAI Responses API (`/v1/responses`) tuki Codexille |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS-sormenjälkien huijaus** | Ohita TLS-pohjainen bot-tunnistus wreq-js:n avulla |
|
||||
| 🌐 **IP-suodatus** | API-käyttöoikeuksien hallinnan sallittu-/estoluettelo |
|
||||
| 📊 **Muokattavat hintarajat** | Konfiguroitava kierrosluku, minimiväli ja suurin samanaikainen järjestelmätasolla |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Todennusportin + tarjoajan esto `/models`-päätepisteelle |
|
||||
| 🔒 **Välityspalvelimen näkyvyys** | Värikoodatut merkit: 🟢 maailmanlaajuinen, 🟡 tarjoaja, 🔵 yhteyskohtainen IP-näytöllä |
|
||||
| 🌐 **3-tason välityspalvelimen määritys** | Määritä välityspalvelimet maailmanlaajuisesti, palveluntarjoajakohtaisesti tai yhteyskohtaisesti |
|
||||
|
||||
+270
-9
@@ -157,6 +157,263 @@ Résultat : Ne jamais arrêter de coder, coût minimal
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Démarrage rapide
|
||||
|
||||
**1. Installer globalement :**
|
||||
@@ -313,6 +570,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Modèles personnalisés** | Ajoutez n'importe quel ID de modèle à n'importe quel fournisseur |
|
||||
| 🌐 **Routeur wildcard** | Routez les patterns `provider/*` vers n'importe quel fournisseur dynamiquement |
|
||||
| 🧠 **Budget de raisonnement** | Modes passthrough, auto, custom et adaptive pour les modèles de raisonnement |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Injection System Prompt** | System prompt global appliqué à toutes les requêtes |
|
||||
| 📄 **API Responses** | Support complet de l'API Responses d'OpenAI (`/v1/responses`) pour Codex |
|
||||
|
||||
@@ -329,15 +588,17 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ Résilience & Sécurité
|
||||
|
||||
| Fonctionnalité | Ce qu'elle fait |
|
||||
| ------------------------------- | -------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Ouverture/fermeture auto par fournisseur avec seuils configurables |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + sémaphore de rate-limit pour les fournisseurs avec clé API |
|
||||
| 🧠 **Cache sémantique** | Cache à deux niveaux (signature + sémantique) réduit coût et latence |
|
||||
| ⚡ **Idempotence des requêtes** | Fenêtre de dédup 5s pour les requêtes dupliquées |
|
||||
| 🔒 **Spoofing TLS Fingerprint** | Contournement de détection de bot via wreq-js |
|
||||
| 🌐 **Filtrage IP** | Allowlist/blocklist pour le contrôle d'accès API |
|
||||
| 📊 **Rate limits éditables** | RPM configurable, intervalle minimum, concurrence max |
|
||||
| Fonctionnalité | Ce qu'elle fait |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Ouverture/fermeture auto par fournisseur avec seuils configurables |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + sémaphore de rate-limit pour les fournisseurs avec clé API |
|
||||
| 🧠 **Cache sémantique** | Cache à deux niveaux (signature + sémantique) réduit coût et latence |
|
||||
| ⚡ **Idempotence des requêtes** | Fenêtre de dédup 5s pour les requêtes dupliquées |
|
||||
| 🔒 **Spoofing TLS Fingerprint** | Contournement de détection de bot via wreq-js |
|
||||
| 🌐 **Filtrage IP** | Allowlist/blocklist pour le contrôle d'accès API |
|
||||
| 📊 **Rate limits éditables** | RPM configurable, intervalle minimum, concurrence max |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
|
||||
### 📊 Observabilité & Analytique
|
||||
|
||||
|
||||
+286
-25
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ התחלה מהירה
|
||||
|
||||
**1. התקן ברחבי העולם:**
|
||||
@@ -300,19 +557,21 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🧠 ניתוב ליבה ומודיעין
|
||||
|
||||
| תכונה | מה זה עושה |
|
||||
| ---------------------------- | -------------------------------------------------------------------------------- |
|
||||
| 🎯 **Smart 4-tier Fallback** | מסלול אוטומטי: מנוי → מפתח API → זול → חינם |
|
||||
| 📊 **מעקב מכסות בזמן אמת** | ספירת אסימונים חיה + איפוס ספירה לאחור לכל ספק |
|
||||
| 🔄 **תרגום פורמט** | OpenAI ↔ קלוד ↔ תאומים ↔ סמן ↔ Kiro חלקה + חיטוי תגובה |
|
||||
| 👥 **תמיכה בריבוי חשבונות** | מספר חשבונות לכל ספק עם בחירה חכמה |
|
||||
| 🔄 **רענון אסימון אוטומטי** | אסימוני OAuth מתרעננים אוטומטית עם ניסיון חוזר |
|
||||
| 🎨 **שילובים מותאמים אישית** | 6 אסטרטגיות: מילוי ראשון, עגול רובין, p2c, אקראי, פחות בשימוש, אופטימיזציה לעלות |
|
||||
| 🧩 **דגמים מותאמים אישית** | הוסף כל מזהה דגם לכל ספק |
|
||||
| 🌐 **נתב תווים כלליים** | נתב דפוסי `provider/*` לכל ספק באופן דינמי |
|
||||
| 🧠 **תקציב חשיבה** | מצבי מעבר, אוטומטי, מותאמים אישית והתאמה למודלים של חשיבה |
|
||||
| 💬 **הזרקת מערכת מיידית** | הודעת מערכת גלובלית הוחלה בכל הבקשות |
|
||||
| 📄 **Responses API** | תמיכה מלאה של OpenAI Responses API (`/v1/responses`) עבור Codex |
|
||||
| תכונה | מה זה עושה |
|
||||
| ----------------------------- | -------------------------------------------------------------------------------- |
|
||||
| 🎯 **Smart 4-tier Fallback** | מסלול אוטומטי: מנוי → מפתח API → זול → חינם |
|
||||
| 📊 **מעקב מכסות בזמן אמת** | ספירת אסימונים חיה + איפוס ספירה לאחור לכל ספק |
|
||||
| 🔄 **תרגום פורמט** | OpenAI ↔ קלוד ↔ תאומים ↔ סמן ↔ Kiro חלקה + חיטוי תגובה |
|
||||
| 👥 **תמיכה בריבוי חשבונות** | מספר חשבונות לכל ספק עם בחירה חכמה |
|
||||
| 🔄 **רענון אסימון אוטומטי** | אסימוני OAuth מתרעננים אוטומטית עם ניסיון חוזר |
|
||||
| 🎨 **שילובים מותאמים אישית** | 6 אסטרטגיות: מילוי ראשון, עגול רובין, p2c, אקראי, פחות בשימוש, אופטימיזציה לעלות |
|
||||
| 🧩 **דגמים מותאמים אישית** | הוסף כל מזהה דגם לכל ספק |
|
||||
| 🌐 **נתב תווים כלליים** | נתב דפוסי `provider/*` לכל ספק באופן דינמי |
|
||||
| 🧠 **תקציב חשיבה** | מצבי מעבר, אוטומטי, מותאמים אישית והתאמה למודלים של חשיבה |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **הזרקת מערכת מיידית** | הודעת מערכת גלובלית הוחלה בכל הבקשות |
|
||||
| 📄 **Responses API** | תמיכה מלאה של OpenAI Responses API (`/v1/responses`) עבור Codex |
|
||||
|
||||
### 🎵 ממשקי API רב-מודאליים
|
||||
|
||||
@@ -327,18 +586,20 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ חוסן וביטחון
|
||||
|
||||
| תכונה | מה זה עושה |
|
||||
| ------------------------------------ | -------------------------------------------------------------- |
|
||||
| 🔌 **מפסק מעגלים** | פתיחה/סגירה אוטומטית לכל ספק עם ערכי סף הניתנים להגדרה |
|
||||
| 🛡️ **עדר נגד רעמים** | Mutex + מגבלת קצב סמפור עבור ספקי מפתח API |
|
||||
| 🧠 **מטמון סמנטי** | מטמון דו-שכבתי (חתימה + סמנטי) מפחית את העלות והשהייה |
|
||||
| ⚡ **בקש אימפוטנציה** | חלון ביטול 5s עבור בקשות כפולות |
|
||||
| 🔒 **זיוף טביעות אצבע TLS** | עוקף זיהוי בוט מבוסס TLS באמצעות wreq-js |
|
||||
| 🌐 **סינון IP** | רשימת הרשאות/רשימת חסימות עבור בקרת גישה ל-API |
|
||||
| 📊 **מגבלות תעריפים הניתנות לעריכה** | RPM ניתן להגדרה, פער מינימלי ומקסימום במקביל ברמת המערכת |
|
||||
| 🛡 **הגנת נקודת קצה API** | Auth gating + חסימת ספק עבור נקודת הקצה `/models` |
|
||||
| 🔒 **נראות פרוקסי** | תגים מקודדים בצבע: 🟢 גלובלי, 🟡 ספק, 🔵 לכל חיבור עם תצוגת IP |
|
||||
| 🌐 **תצורת פרוקסי בשלוש רמות** | הגדר פרוקסי ברמה גלובלית, לכל ספק או לכל חיבור |
|
||||
| תכונה | מה זה עושה |
|
||||
| ------------------------------------ | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **מפסק מעגלים** | פתיחה/סגירה אוטומטית לכל ספק עם ערכי סף הניתנים להגדרה |
|
||||
| 🛡️ **עדר נגד רעמים** | Mutex + מגבלת קצב סמפור עבור ספקי מפתח API |
|
||||
| 🧠 **מטמון סמנטי** | מטמון דו-שכבתי (חתימה + סמנטי) מפחית את העלות והשהייה |
|
||||
| ⚡ **בקש אימפוטנציה** | חלון ביטול 5s עבור בקשות כפולות |
|
||||
| 🔒 **זיוף טביעות אצבע TLS** | עוקף זיהוי בוט מבוסס TLS באמצעות wreq-js |
|
||||
| 🌐 **סינון IP** | רשימת הרשאות/רשימת חסימות עבור בקרת גישה ל-API |
|
||||
| 📊 **מגבלות תעריפים הניתנות לעריכה** | RPM ניתן להגדרה, פער מינימלי ומקסימום במקביל ברמת המערכת |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **הגנת נקודת קצה API** | Auth gating + חסימת ספק עבור נקודת הקצה `/models` |
|
||||
| 🔒 **נראות פרוקסי** | תגים מקודדים בצבע: 🟢 גלובלי, 🟡 ספק, 🔵 לכל חיבור עם תצוגת IP |
|
||||
| 🌐 **תצורת פרוקסי בשלוש רמות** | הגדר פרוקסי ברמה גלובלית, לכל ספק או לכל חיבור |
|
||||
|
||||
### 📊 יכולת תצפית וניתוח
|
||||
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Gyors kezdés
|
||||
|
||||
**1. Globális telepítés:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Egyedi modellek** | Adjon hozzá bármilyen modellazonosítót bármely szolgáltatóhoz |
|
||||
| 🌐 **Wildcard Router** | `provider/*` minták továbbítása bármely szolgáltatóhoz dinamikusan |
|
||||
| 🧠 **Átgondolt költségvetés** | Átjárási, automatikus, egyéni és adaptív módok érvelési modellekhez |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Rendszer azonnali befecskendezés** | Globális rendszerkérdés minden kérelemre érvényes |
|
||||
| 📄 **Responses API** | Teljes OpenAI Responses API (`/v1/responses`) támogatás a Codexhez |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS ujjlenyomat-hamisítás** | A TLS-alapú botészlelés megkerülése a wreq-js segítségével |
|
||||
| 🌐 **IP-szűrés** | Allowlist/blokkolista API hozzáférés-vezérléshez |
|
||||
| 📊 **Szerkeszthető díjkorlátok** | Konfigurálható fordulatszám, minimális rés és maximális egyidejű rendszerszinten |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Auth kapuzás + szolgáltató blokkolása a `/models` végponthoz |
|
||||
| 🔒 **Proxy láthatósága** | Színkódolt jelvények: 🟢 globális, 🟡 szolgáltató, 🔵 kapcsolatonként IP kijelzővel |
|
||||
| 🌐 **3-szintű proxykonfiguráció** | Proxyk konfigurálása globális, szolgáltatónkénti vagy kapcsolatonkénti szinten |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mulai Cepat
|
||||
|
||||
**1. Instal secara global:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Model Khusus** | Tambahkan ID model apa pun ke penyedia mana pun |
|
||||
| 🌐 **Router Wildcard** | Rutekan pola `provider/*` ke penyedia mana pun secara dinamis |
|
||||
| 🧠 **Memikirkan Anggaran** | Mode passthrough, otomatis, kustom, dan adaptif untuk model penalaran |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Injeksi Perintah Sistem** | Perintah sistem global diterapkan di semua permintaan |
|
||||
| 📄 **API Respons** | Dukungan penuh OpenAI Responses API (`/v1/responses`) untuk Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **Spoofing Sidik Jari TLS** | Lewati deteksi bot berbasis TLS melalui wreq-js |
|
||||
| 🌐 **Pemfilteran IP** | Daftar yang diizinkan/daftar blokir untuk kontrol akses API |
|
||||
| 📊 **Batas Tarif yang Dapat Diedit** | RPM yang dapat dikonfigurasi, celah minimum, dan maks secara bersamaan pada tingkat sistem |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Perlindungan Titik Akhir API** | Gerbang autentikasi + pemblokiran penyedia untuk titik akhir `/models` |
|
||||
| 🔒 **Visibilitas Proksi** | Lencana berkode warna: 🟢 global, 🟡 penyedia, 🔵 per koneksi dengan tampilan IP |
|
||||
| 🌐 **Konfigurasi Proksi 3 Tingkat** | Konfigurasikan proxy di tingkat global, per penyedia, atau per koneksi |
|
||||
|
||||
+261
@@ -80,6 +80,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ त्वरित शुरुआत
|
||||
|
||||
**1. विश्व स्तर पर स्थापित करें:**
|
||||
@@ -208,6 +465,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **कस्टम मॉडल** | किसी भी प्रदाता से कोई भी मॉडल आईडी जोड़ें |
|
||||
| 🌐 **वाइल्डकार्ड राउटर** | `provider/*` पैटर्न को गतिशील रूप से किसी भी प्रदाता तक रूट करें |
|
||||
| 🧠 **सोच बजट** | तर्क मॉडल के लिए पासथ्रू, ऑटो, कस्टम और अनुकूली मोड |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **सिस्टम प्रॉम्प्ट इंजेक्शन** | ग्लोबल सिस्टम प्रॉम्प्ट सभी अनुरोधों पर लागू किया गया |
|
||||
| 📄 **प्रतिक्रियाएं एपीआई** | कोडेक्स के लिए पूर्ण ओपनएआई रिस्पॉन्स एपीआई (`/v1/responses`) समर्थन |
|
||||
|
||||
@@ -233,6 +492,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **टीएलएस फ़िंगरप्रिंट स्पूफ़िंग** | Wreq-js के माध्यम से टीएलएस-आधारित बॉट डिटेक्शन को बायपास करें |
|
||||
| 🌐 **आईपी फ़िल्टरिंग** | एपीआई अभिगम नियंत्रण के लिए अनुमति सूची/अवरुद्ध सूची |
|
||||
| 📊 **संपादन योग्य दर सीमाएँ** | सिस्टम स्तर पर कॉन्फ़िगर करने योग्य आरपीएम, न्यूनतम अंतर और अधिकतम समवर्ती |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **एपीआई एंडपॉइंट सुरक्षा** | `/models` समापन बिंदु के लिए ऑथेंटिक गेटिंग + प्रदाता अवरोधन |
|
||||
| 🔒 **प्रॉक्सी दृश्यता** | रंग-कोडित बैज: 🟢 वैश्विक, 🟡 प्रदाता, 🔵 आईपी डिस्प्ले के साथ प्रति-कनेक्शन |
|
||||
| 🌐 **3-स्तरीय प्रॉक्सी कॉन्फ़िगरेशन** | वैश्विक, प्रति-प्रदाता, या प्रति-कनेक्शन स्तर पर प्रॉक्सी कॉन्फ़िगर करें |
|
||||
|
||||
+270
-9
@@ -157,6 +157,263 @@ Risultato: Non smettere mai di programmare, costo minimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Avvio Rapido
|
||||
|
||||
**1. Installa globalmente:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Modelli personalizzati** | Aggiungi qualsiasi ID modello a qualsiasi provider |
|
||||
| 🌐 **Router wildcard** | Instrada pattern `provider/*` verso qualsiasi provider dinamicamente |
|
||||
| 🧠 **Budget di ragionamento** | Modalità passthrough, auto, custom e adaptive per modelli di ragionamento |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Iniezione System Prompt** | System prompt globale applicato a tutte le richieste |
|
||||
| 📄 **API Responses** | Supporto completo per OpenAI Responses API (`/v1/responses`) per Codex |
|
||||
|
||||
@@ -327,15 +586,17 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ Resilienza & Sicurezza
|
||||
|
||||
| Funzionalità | Cosa Fa |
|
||||
| ------------------------------- | -------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Apertura/chiusura auto per provider con soglie configurabili |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semaforo rate-limit per provider con API key |
|
||||
| 🧠 **Cache semantica** | Cache a due livelli (firma + semantica) riduce costi e latenza |
|
||||
| ⚡ **Idempotenza richieste** | Finestra dedup 5s per richieste duplicate |
|
||||
| 🔒 **Spoofing TLS Fingerprint** | Bypass rilevamento bot tramite wreq-js |
|
||||
| 🌐 **Filtro IP** | Allowlist/blocklist per controllo accesso API |
|
||||
| 📊 **Rate limit modificabili** | RPM, gap minimo e concorrenza massima configurabili |
|
||||
| Funzionalità | Cosa Fa |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Apertura/chiusura auto per provider con soglie configurabili |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semaforo rate-limit per provider con API key |
|
||||
| 🧠 **Cache semantica** | Cache a due livelli (firma + semantica) riduce costi e latenza |
|
||||
| ⚡ **Idempotenza richieste** | Finestra dedup 5s per richieste duplicate |
|
||||
| 🔒 **Spoofing TLS Fingerprint** | Bypass rilevamento bot tramite wreq-js |
|
||||
| 🌐 **Filtro IP** | Allowlist/blocklist per controllo accesso API |
|
||||
| 📊 **Rate limit modificabili** | RPM, gap minimo e concorrenza massima configurabili |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
|
||||
### 📊 Osservabilità & Analytics
|
||||
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ クイックスタート
|
||||
|
||||
**1.グローバルにインストール:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **カスタムモデル** | 任意のモデル ID を任意のプロバイダーに追加する |
|
||||
| 🌐 **ワイルドカードルーター** | `provider/*` パターンを任意のプロバイダーに動的にルーティングする |
|
||||
| 🧠 **予算を考える** | 推論モデルのパススルー、自動、カスタム、および適応モード |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **システム プロンプト インジェクション** | すべてのリクエストに適用されるグローバル システム プロンプト |
|
||||
| 📄 **レスポンス API** | Codex の OpenAI Response API (`/v1/responses`) の完全なサポート |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS 指紋スプーフィング** | wreq-js 経由で TLS ベースのボット検出をバイパスする |
|
||||
| 🌐 **IP フィルタリング** | API アクセス制御の許可リスト/ブロックリスト |
|
||||
| 📊 **編集可能なレート制限** | システム レベルで構成可能な RPM、最小ギャップ、最大同時実行 |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API エンドポイント保護** | `/models` エンドポイントの認証ゲート + プロバイダー ブロック |
|
||||
| 🔒 **プロキシの可視性** | 色分けされたバッジ: 🟢 グローバル、🟡 プロバイダー、🔵 IP 表示による接続ごと |
|
||||
| 🌐 **3 レベルのプロキシ構成** | グローバル、プロバイダーごと、または接続ごとのレベルでプロキシを構成する |
|
||||
|
||||
+286
-25
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ 빠른 시작
|
||||
|
||||
**1. 전역적으로 설치:**
|
||||
@@ -300,19 +557,21 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🧠 코어 라우팅 및 인텔리전스
|
||||
|
||||
| 기능 | 그것이 하는 일 |
|
||||
| --------------------------- | ------------------------------------------------------------------------- |
|
||||
| 🎯 **스마트 4계층 폴백** | 자동 경로: 구독 → API 키 → 저렴한 → 무료 |
|
||||
| 📊 **실시간 할당량 추적** | 라이브 토큰 수 + 공급자별 카운트다운 재설정 |
|
||||
| 🔄 **형식 번역** | OpenAI ⇔ Claude ⇔ Gemini ⇔ Cursor ⇔ Kiro 심리스 + 반응위생 |
|
||||
| 👥 **다중 계정 지원** | 지능적인 선택을 통해 공급자당 여러 계정 |
|
||||
| 🔄 **자동 토큰 새로고침** | 재시도 시 OAuth 토큰이 자동으로 새로 고쳐집니다. |
|
||||
| 🎨 **맞춤형 콤보** | 6가지 전략: 채우기 우선, 라운드 로빈, p2c, 무작위, 최소 사용, 비용 최적화 |
|
||||
| 🧩 **맞춤형 모델** | 모든 공급자에 모델 ID 추가 |
|
||||
| 🌐 **와일드카드 라우터** | `provider/*` 패턴을 모든 공급자에게 동적으로 라우팅 |
|
||||
| 🧠 **예산 고려** | 추론 모델을 위한 패스스루, 자동, 사용자 정의 및 적응형 모드 |
|
||||
| 💬 **시스템 프롬프트 삽입** | 모든 요청에 전역 시스템 프롬프트 적용 |
|
||||
| 📄 **응답 API** | Codex에 대한 전체 OpenAI 응답 API(`/v1/responses`) 지원 |
|
||||
| 기능 | 그것이 하는 일 |
|
||||
| ----------------------------- | ----------------------------------------------------------------------------- |
|
||||
| 🎯 **스마트 4계층 폴백** | 자동 경로: 구독 → API 키 → 저렴한 → 무료 |
|
||||
| 📊 **실시간 할당량 추적** | 라이브 토큰 수 + 공급자별 카운트다운 재설정 |
|
||||
| 🔄 **형식 번역** | OpenAI ⇔ Claude ⇔ Gemini ⇔ Cursor ⇔ Kiro 심리스 + 반응위생 |
|
||||
| 👥 **다중 계정 지원** | 지능적인 선택을 통해 공급자당 여러 계정 |
|
||||
| 🔄 **자동 토큰 새로고침** | 재시도 시 OAuth 토큰이 자동으로 새로 고쳐집니다. |
|
||||
| 🎨 **맞춤형 콤보** | 6가지 전략: 채우기 우선, 라운드 로빈, p2c, 무작위, 최소 사용, 비용 최적화 |
|
||||
| 🧩 **맞춤형 모델** | 모든 공급자에 모델 ID 추가 |
|
||||
| 🌐 **와일드카드 라우터** | `provider/*` 패턴을 모든 공급자에게 동적으로 라우팅 |
|
||||
| 🧠 **예산 고려** | 추론 모델을 위한 패스스루, 자동, 사용자 정의 및 적응형 모드 |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **시스템 프롬프트 삽입** | 모든 요청에 전역 시스템 프롬프트 적용 |
|
||||
| 📄 **응답 API** | Codex에 대한 전체 OpenAI 응답 API(`/v1/responses`) 지원 |
|
||||
|
||||
### 🎵 다중 모드 API
|
||||
|
||||
@@ -327,18 +586,20 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ 복원력 및 보안
|
||||
|
||||
| 기능 | 그것이 하는 일 |
|
||||
| ---------------------------- | -------------------------------------------------------------------------- |
|
||||
| 🔌 **회로 차단기** | 구성 가능한 임계값을 사용하여 공급자별 자동 열기/닫기 |
|
||||
| 🛡️ **천둥 방지 무리** | API 키 공급자에 대한 뮤텍스 + 세마포 속도 제한 |
|
||||
| 🧠 **의미론적 캐시** | 2계층 캐시(서명 + 의미 체계)로 비용 및 대기 시간 감소 |
|
||||
| ⚡ **멱등성 요청** | 중복 요청에 대한 5초 중복 제거 기간 |
|
||||
| 🔒 **TLS 지문 스푸핑** | wreq-js를 통해 TLS 기반 봇 감지 우회 |
|
||||
| 🌐 **IP 필터링** | API 액세스 제어를 위한 허용 목록/차단 목록 |
|
||||
| 📊 **편집 가능한 속도 제한** | 시스템 수준에서 구성 가능한 RPM, 최소 간격 및 최대 동시 |
|
||||
| 🛡 **API 엔드포인트 보호** | `/models` 엔드포인트에 대한 인증 게이팅 + 공급자 차단 |
|
||||
| 🔒 **프록시 가시성** | 색상으로 구분된 배지: 🟢 글로벌, 🟡 공급자, 🔵 IP 디스플레이를 통한 연결별 |
|
||||
| 🌐 **3단계 프록시 구성** | 글로벌, 공급자별 또는 연결별 수준에서 프록시 구성 |
|
||||
| 기능 | 그것이 하는 일 |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **회로 차단기** | 구성 가능한 임계값을 사용하여 공급자별 자동 열기/닫기 |
|
||||
| 🛡️ **천둥 방지 무리** | API 키 공급자에 대한 뮤텍스 + 세마포 속도 제한 |
|
||||
| 🧠 **의미론적 캐시** | 2계층 캐시(서명 + 의미 체계)로 비용 및 대기 시간 감소 |
|
||||
| ⚡ **멱등성 요청** | 중복 요청에 대한 5초 중복 제거 기간 |
|
||||
| 🔒 **TLS 지문 스푸핑** | wreq-js를 통해 TLS 기반 봇 감지 우회 |
|
||||
| 🌐 **IP 필터링** | API 액세스 제어를 위한 허용 목록/차단 목록 |
|
||||
| 📊 **편집 가능한 속도 제한** | 시스템 수준에서 구성 가능한 RPM, 최소 간격 및 최대 동시 |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API 엔드포인트 보호** | `/models` 엔드포인트에 대한 인증 게이팅 + 공급자 차단 |
|
||||
| 🔒 **프록시 가시성** | 색상으로 구분된 배지: 🟢 글로벌, 🟡 공급자, 🔵 IP 디스플레이를 통한 연결별 |
|
||||
| 🌐 **3단계 프록시 구성** | 글로벌, 공급자별 또는 연결별 수준에서 프록시 구성 |
|
||||
|
||||
### 📊 관찰 가능성 및 분석
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
|
||||
_Your universal API proxy — one endpoint, 36+ providers, zero downtime._
|
||||
|
||||
**Chat Completions • Embeddings • Image Generation • Audio • Reranking • 100% TypeScript**
|
||||
**Chat Completions • Embeddings • Image Generation • Video • Music • Audio • Reranking • 100% TypeScript**
|
||||
|
||||
---
|
||||
|
||||
@@ -32,7 +32,7 @@ _Connect any AI-powered IDE or CLI tool through OmniRoute — free API gateway f
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center" width="110">
|
||||
<a href="https://github.com/cline/cline">
|
||||
<a href="https://github.com/openclaw/openclaw">
|
||||
<img src="./public/providers/openclaw.png" alt="OpenClaw" width="48"/><br/>
|
||||
<b>OpenClaw</b>
|
||||
</a><br/>
|
||||
@@ -169,6 +169,267 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 10 providers and 20+ models (OpenAI, xAI, Together, Fireworks, Nebius, Hyperbolic, NanoBanana, Antigravity, SD WebUI, ComfyUI)
|
||||
- **Text-to-Video** — `/v1/videos/generations` — ComfyUI (AnimateDiff, SVD) and SD WebUI
|
||||
- **Text-to-Music** — `/v1/music/generations` — ComfyUI (Stable Audio Open, MusicGen)
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper + Nvidia NIM, HuggingFace, Qwen3
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — ElevenLabs, Nvidia NIM, HuggingFace, Coqui, Tortoise, Qwen3, + existing providers
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
**1. Install globally:**
|
||||
@@ -346,19 +607,23 @@ When minimized, OmniRoute lives in your system tray with quick actions:
|
||||
| 🧩 **Custom Models** | Add any model ID to any provider |
|
||||
| 🌐 **Wildcard Router** | Route `provider/*` patterns to any provider dynamically |
|
||||
| 🧠 **Thinking Budget** | Passthrough, auto, custom, and adaptive modes for reasoning models |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Injection** | Global system prompt applied across all requests |
|
||||
| 📄 **Responses API** | Full OpenAI Responses API (`/v1/responses`) support for Codex |
|
||||
|
||||
### 🎵 Multi-Modal APIs
|
||||
|
||||
| Feature | What It Does |
|
||||
| -------------------------- | --------------------------------------------------- |
|
||||
| 🖼️ **Image Generation** | `/v1/images/generations` — 4 providers, 9+ models |
|
||||
| 📐 **Embeddings** | `/v1/embeddings` — 6 providers, 9+ models |
|
||||
| 🎤 **Audio Transcription** | `/v1/audio/transcriptions` — Whisper-compatible |
|
||||
| 🔊 **Text-to-Speech** | `/v1/audio/speech` — Multi-provider audio synthesis |
|
||||
| 🛡️ **Moderations** | `/v1/moderations` — Content safety checks |
|
||||
| 🔀 **Reranking** | `/v1/rerank` — Document relevance reranking |
|
||||
| Feature | What It Does |
|
||||
| -------------------------- | -------------------------------------------------------------------------------- |
|
||||
| 🖼️ **Image Generation** | `/v1/images/generations` — 10 providers, 20+ models (cloud + local) |
|
||||
| 📐 **Embeddings** | `/v1/embeddings` — 6 providers, 9+ models |
|
||||
| 🎤 **Audio Transcription** | `/v1/audio/transcriptions` — Whisper + Nvidia NIM, HuggingFace, Qwen3 |
|
||||
| 🔊 **Text-to-Speech** | `/v1/audio/speech` — ElevenLabs, Nvidia NIM, HuggingFace, Coqui, Tortoise, Qwen3 |
|
||||
| 🎬 **Video Generation** | `/v1/videos/generations` — ComfyUI (AnimateDiff, SVD), SD WebUI |
|
||||
| 🎵 **Music Generation** | `/v1/music/generations` — ComfyUI (Stable Audio Open, MusicGen) |
|
||||
| 🛡️ **Moderations** | `/v1/moderations` — Content safety checks |
|
||||
| 🔀 **Reranking** | `/v1/rerank` — Document relevance reranking |
|
||||
|
||||
### 🛡️ Resilience & Security
|
||||
|
||||
@@ -371,6 +636,8 @@ When minimized, OmniRoute lives in your system tray with quick actions:
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Bypass TLS-based bot detection via wreq-js |
|
||||
| 🌐 **IP Filtering** | Allowlist/blocklist for API access control |
|
||||
| 📊 **Editable Rate Limits** | Configurable RPM, min gap, and max concurrent at system level |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Auth gating + provider blocking for the `/models` endpoint |
|
||||
| 🔒 **Proxy Visibility** | Color-coded badges: 🟢 global, 🟡 provider, 🔵 per-connection with IP display |
|
||||
| 🌐 **3-Level Proxy Config** | Configure proxies at global, per-provider, or per-connection level |
|
||||
@@ -828,6 +1095,47 @@ Settings → API Configuration:
|
||||
Model: if/kimi-k2-thinking
|
||||
```
|
||||
|
||||
### OpenCode
|
||||
|
||||
**Step 1:** Add OmniRoute as a custom provider:
|
||||
|
||||
```bash
|
||||
opencode
|
||||
/connect
|
||||
# Select "Other" → Enter ID: "omniroute" → Enter your OmniRoute API key
|
||||
```
|
||||
|
||||
**Step 2:** Create/edit `opencode.json` in your project root:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"omniroute": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "OmniRoute",
|
||||
"options": {
|
||||
"baseURL": "http://localhost:20128/v1"
|
||||
},
|
||||
"models": {
|
||||
"cc/claude-sonnet-4-20250514": { "name": "Claude Sonnet 4" },
|
||||
"gg/gemini-2.5-pro": { "name": "Gemini 2.5 Pro" },
|
||||
"if/kimi-k2-thinking": { "name": "Kimi K2 (Free)" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 3:** Select the model in OpenCode:
|
||||
|
||||
```bash
|
||||
/models
|
||||
# Select any OmniRoute model from the list
|
||||
```
|
||||
|
||||
> **Tip:** Add any model available in your OmniRoute `/v1/models` endpoint to the `models` section. Use the format `provider/model-id` from your OmniRoute dashboard.
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mula Pantas
|
||||
|
||||
**1. Pasang secara global:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Model Tersuai** | Tambahkan sebarang ID model pada mana-mana pembekal |
|
||||
| 🌐 **Penghala Wildcard** | Halakan corak `provider/*` kepada mana-mana pembekal secara dinamik |
|
||||
| 🧠 **Anggaran Berfikir** | Mod laluan, auto, tersuai dan adaptif untuk model penaakulan |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Suntikan** | Gesaan sistem global digunakan merentas semua permintaan |
|
||||
| 📄 **API Respons** | Sokongan OpenAI Responses API (`/v1/responses`) penuh untuk Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Pintas pengesanan bot berasaskan TLS melalui wreq-js |
|
||||
| 🌐 **Penapisan IP** | Senarai kebenaran/senarai sekat untuk kawalan akses API |
|
||||
| 📊 **Had Kadar Boleh Diedit** | RPM boleh dikonfigurasikan, jurang min dan serentak maksimum pada tahap sistem |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Perlindungan Titik Akhir API** | Gating pengesahan + penyekatan penyedia untuk titik akhir `/models` |
|
||||
| 🔒 **Keterlihatan Proksi** | Lencana berkod warna: 🟢 global, 🟡 pembekal, 🔵 setiap sambungan dengan paparan IP |
|
||||
| 🌐 **Konfigurasi Proksi 3 Tahap** | Konfigurasikan proksi pada peringkat global, setiap pembekal atau setiap sambungan |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Snelle start
|
||||
|
||||
**1. Wereldwijd installeren:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Aangepaste modellen** | Voeg elke model-ID toe aan elke provider |
|
||||
| 🌐 **Wildcard-router** | Stuur `provider/*` patronen dynamisch naar elke provider |
|
||||
| 🧠 **Denkbudget** | Passthrough-, automatische, aangepaste en adaptieve modi voor redeneermodellen |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Systeempromptinjectie** | Algemene systeemprompt toegepast op alle verzoeken |
|
||||
| 📄 **Reacties-API** | Volledige OpenAI Responses API (`/v1/responses`) ondersteuning voor Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS-vingerafdrukspoofing** | Omzeil TLS-gebaseerde botdetectie via wreq-js |
|
||||
| 🌐 **IP-filtering** | Toelatingslijst/blokkeerlijst voor API-toegangscontrole |
|
||||
| 📊 **Bewerkbare tarieflimieten** | Configureerbare RPM, minimale tussenruimte en maximale gelijktijdigheid op systeemniveau |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API-eindpuntbescherming** | Auth-gating + providerblokkering voor het `/models` eindpunt |
|
||||
| 🔒 **Proxyzichtbaarheid** | Kleurgecodeerde badges: 🟢 wereldwijd, 🟡 provider, 🔵 per verbinding met IP-display |
|
||||
| 🌐 **Proxyconfiguratie op 3 niveaus** | Configureer proxy's op globaal, per provider of per verbindingsniveau |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Hurtigstart
|
||||
|
||||
**1. Installer globalt:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Egendefinerte modeller** | Legg til hvilken som helst modell-ID til en hvilken som helst leverandør |
|
||||
| 🌐 **Wildcard-ruter** | Ruter `provider/*`-mønstre til enhver leverandør dynamisk |
|
||||
| 🧠 **Tenkebudsjett** | Passthrough, auto, egendefinerte og adaptive moduser for resonnerende modeller |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Systemprompt-injeksjon** | Global systemforespørsel brukt på alle forespørsler |
|
||||
| 📄 **Responses API** | Full støtte for OpenAI Responses API (`/v1/responses`) for Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS-fingeravtrykkspoofing** | Omgå TLS-basert botdeteksjon via wreq-js |
|
||||
| 🌐 **IP-filtrering** | Tillatelsesliste/blokkeringsliste for API-tilgangskontroll |
|
||||
| 📊 **Redigerbare satsgrenser** | Konfigurerbar RPM, min gap og maks samtidig på systemnivå |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API-endepunktbeskyttelse** | Auth-gate + leverandørblokkering for `/models`-endepunktet |
|
||||
| 🔒 **Proxy-synlighet** | Fargekodede merker: 🟢 global, 🟡 leverandør, 🔵 per tilkobling med IP-skjerm |
|
||||
| 🌐 **3-nivå proxy-konfigurasjon** | Konfigurer proxyer på globalt nivå, per leverandør eller per tilkoblingsnivå |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Mabilis na Pagsisimula
|
||||
|
||||
**1. I-install sa buong mundo:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Mga Custom na Modelo** | Magdagdag ng anumang ID ng modelo sa anumang provider |
|
||||
| 🌐 **Wildcard Router** | Iruta ang `provider/*` na mga pattern sa anumang provider nang dynamic na |
|
||||
| 🧠 **Badyet sa Pag-iisip** | Passthrough, auto, custom, at adaptive mode para sa mga modelo ng pangangatwiran |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Injection** | Inilapat ang global system prompt sa lahat ng kahilingan |
|
||||
| 📄 **Responses API** | Buong OpenAI Responses API (`/v1/responses`) na suporta para sa Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS Fingerprint Spoofing** | I-bypass ang TLS-based na bot detection sa pamamagitan ng wreq-js |
|
||||
| 🌐 **Pag-filter ng IP** | Allowlist/blocklist para sa API access control |
|
||||
| 📊 **Mga Nae-edit na Limitasyon sa Rate** | Configurable RPM, min gap, at max na kasabay sa antas ng system |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Proteksyon sa Endpoint ng API** | Auth gating + pagharang ng provider para sa `/models` endpoint |
|
||||
| 🔒 **Proxy Visibility** | Mga color-coded na badge: 🟢 global, 🟡 provider, 🔵 per-connection na may IP display |
|
||||
| 🌐 **3-Level Proxy Config** | I-configure ang mga proxy sa global, per-provider, o per-connection level |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Szybki start
|
||||
|
||||
**1. Zainstaluj globalnie:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Modele niestandardowe** | Dodaj dowolny identyfikator modelu do dowolnego dostawcy |
|
||||
| 🌐 **Router z dziką kartą** | Dynamicznie kieruj `provider/*` wzorce do dowolnego dostawcy |
|
||||
| 🧠 **Myślący budżet** | Tryby przekazywania, automatyczne, niestandardowe i adaptacyjne dla modeli wnioskowania |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Wstrzyknięcie monitu systemowego** | Globalny monit systemowy stosowany do wszystkich żądań |
|
||||
| 📄 **API odpowiedzi** | Pełna obsługa OpenAI Responses API (`/v1/responses`) dla Codexu |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **Podrabianie odcisków palców TLS** | Pomiń wykrywanie botów opartych na TLS poprzez wreq-js |
|
||||
| 🌐 **Filtrowanie IP** | Lista dozwolonych/blokowanych dla kontroli dostępu API |
|
||||
| 📊 **Edytowalne limity stawek** | Konfigurowalne obroty, minimalna przerwa i maksymalna równowaga na poziomie systemu |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Ochrona punktu końcowego API** | Bramkowanie uwierzytelniania + blokowanie dostawcy dla punktu końcowego `/models` |
|
||||
| 🔒 **Widoczność proxy** | Oznaczone kolorami plakietki: 🟢 globalny, 🟡 dostawca, 🔵 na połączenie z wyświetlaczem IP |
|
||||
| 🌐 **3-poziomowa konfiguracja serwera proxy** | Skonfiguruj serwery proxy na poziomie globalnym, dla dostawcy lub dla połączenia |
|
||||
|
||||
+299
-34
@@ -7,7 +7,7 @@
|
||||
|
||||
_Seu proxy de API universal — um endpoint, 36+ provedores, zero tempo de inatividade._
|
||||
|
||||
**Chat Completions • Embeddings • Geração de Imagem • Áudio • Reranking • 100% TypeScript**
|
||||
**Chat Completions • Embeddings • Geração de Imagem • Vídeo • Música • Áudio • Reranking • 100% TypeScript**
|
||||
|
||||
---
|
||||
|
||||
@@ -157,6 +157,265 @@ Resultado: Nunca pare de programar, custo mínimo
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 10 providers and 20+ models (OpenAI, xAI, Together, Fireworks, Nebius, Hyperbolic, NanoBanana, Antigravity, SD WebUI, ComfyUI)
|
||||
- **Text-to-Video** — `/v1/videos/generations` — ComfyUI (AnimateDiff, SVD) and SD WebUI
|
||||
- **Text-to-Music** — `/v1/music/generations` — ComfyUI (Stable Audio Open, MusicGen)
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper + Nvidia NIM, HuggingFace, Qwen3
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — ElevenLabs, Nvidia NIM, HuggingFace, Coqui, Tortoise, Qwen3
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Início Rápido
|
||||
|
||||
**1. Instale globalmente:**
|
||||
@@ -315,45 +574,51 @@ Quando minimizado, o OmniRoute fica na bandeja do sistema com ações rápidas:
|
||||
|
||||
### 🧠 Roteamento e Inteligência
|
||||
|
||||
| Funcionalidade | O que Faz |
|
||||
| ----------------------------------------- | ------------------------------------------------------------------------------- |
|
||||
| 🎯 **Fallback Inteligente 4 Tiers** | Auto-roteamento: Assinatura → API Key → Barato → Gratuito |
|
||||
| 📊 **Rastreamento de Cota em Tempo Real** | Contagem de tokens ao vivo + countdown de reset por provedor |
|
||||
| 🔄 **Tradução de Formato** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro transparente |
|
||||
| 👥 **Suporte Multi-Conta** | Múltiplas contas por provedor com seleção inteligente |
|
||||
| 🔄 **Renovação Automática de Token** | Tokens OAuth renovam automaticamente com retry |
|
||||
| 🎨 **Combos Personalizados** | 6 estratégias: fill-first, round-robin, p2c, random, least-used, cost-optimized |
|
||||
| 🧩 **Modelos Personalizados** | Adicione qualquer ID de modelo a qualquer provedor |
|
||||
| 🌐 **Roteador Wildcard** | Roteie padrões `provider/*` para qualquer provedor dinamicamente |
|
||||
| 🧠 **Budget de Raciocínio** | Modos passthrough, auto, custom e adaptativo para modelos de raciocínio |
|
||||
| 💬 **Injeção de System Prompt** | System prompt global aplicado em todas as requisições |
|
||||
| 📄 **API Responses** | Suporte completo à API Responses da OpenAI (`/v1/responses`) para Codex |
|
||||
| Funcionalidade | O que Faz |
|
||||
| ----------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| 🎯 **Fallback Inteligente 4 Tiers** | Auto-roteamento: Assinatura → API Key → Barato → Gratuito |
|
||||
| 📊 **Rastreamento de Cota em Tempo Real** | Contagem de tokens ao vivo + countdown de reset por provedor |
|
||||
| 🔄 **Tradução de Formato** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro transparente |
|
||||
| 👥 **Suporte Multi-Conta** | Múltiplas contas por provedor com seleção inteligente |
|
||||
| 🔄 **Renovação Automática de Token** | Tokens OAuth renovam automaticamente com retry |
|
||||
| 🎨 **Combos Personalizados** | 6 estratégias: fill-first, round-robin, p2c, random, least-used, cost-optimized |
|
||||
| 🧩 **Modelos Personalizados** | Adicione qualquer ID de modelo a qualquer provedor |
|
||||
| 🌐 **Roteador Wildcard** | Roteie padrões `provider/*` para qualquer provedor dinamicamente |
|
||||
| 🧠 **Budget de Raciocínio** | Modos passthrough, auto, custom e adaptativo para modelos de raciocínio |
|
||||
| � **Aliases de Modelo** | Redireciona IDs de modelos depreciados para substitutos atuais (built-in + custom) |
|
||||
| ⚡ **Degradação em Background** | Redireciona tarefas em background (títulos, resumos) para modelos mais baratos |
|
||||
| �💬 **Injeção de System Prompt** | System prompt global aplicado em todas as requisições |
|
||||
| 📄 **API Responses** | Suporte completo à API Responses da OpenAI (`/v1/responses`) para Codex |
|
||||
|
||||
### 🎵 APIs Multi-Modal
|
||||
|
||||
| Funcionalidade | O que Faz |
|
||||
| --------------------------- | ---------------------------------------------------- |
|
||||
| 🖼️ **Geração de Imagem** | `/v1/images/generations` — 4 provedores, 9+ modelos |
|
||||
| 📐 **Embeddings** | `/v1/embeddings` — 6 provedores, 9+ modelos |
|
||||
| 🎤 **Transcrição de Áudio** | `/v1/audio/transcriptions` — Compatível com Whisper |
|
||||
| 🔊 **Texto para Fala** | `/v1/audio/speech` — Síntese de áudio multi-provedor |
|
||||
| 🛡️ **Moderações** | `/v1/moderations` — Verificações de segurança |
|
||||
| 🔀 **Reranking** | `/v1/rerank` — Reranking de relevância de documentos |
|
||||
| Funcionalidade | O que Faz |
|
||||
| --------------------------- | -------------------------------------------------------------------------------- |
|
||||
| 🖼️ **Geração de Imagem** | `/v1/images/generations` — 10 provedores, 20+ modelos (cloud + local) |
|
||||
| 📐 **Embeddings** | `/v1/embeddings` — 6 provedores, 9+ modelos |
|
||||
| 🎤 **Transcrição de Áudio** | `/v1/audio/transcriptions` — Whisper + Nvidia NIM, HuggingFace, Qwen3 |
|
||||
| 🔊 **Texto para Fala** | `/v1/audio/speech` — ElevenLabs, Nvidia NIM, HuggingFace, Coqui, Tortoise, Qwen3 |
|
||||
| 🎬 **Geração de Vídeo** | `/v1/videos/generations` — ComfyUI (AnimateDiff, SVD), SD WebUI |
|
||||
| 🎵 **Geração de Música** | `/v1/music/generations` — ComfyUI (Stable Audio Open, MusicGen) |
|
||||
| 🛡️ **Moderações** | `/v1/moderations` — Verificações de segurança |
|
||||
| 🔀 **Reranking** | `/v1/rerank` — Reranking de relevância de documentos |
|
||||
|
||||
### 🛡️ Resiliência e Segurança
|
||||
|
||||
| Funcionalidade | O que Faz |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Auto-abertura/fechamento por provedor com limites configuráveis |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semáforo rate-limit para provedores com API key |
|
||||
| 🧠 **Cache Semântico** | Cache de duas camadas (assinatura + semântico) reduz custo e latência |
|
||||
| ⚡ **Idempotência de Requisição** | Janela de dedup de 5s para requisições duplicadas |
|
||||
| 🔒 **Spoofing de Fingerprint TLS** | Bypass de detecção de bot via TLS com wreq-js |
|
||||
| 🌐 **Filtragem de IP** | Allowlist/blocklist para controle de acesso à API |
|
||||
| 📊 **Rate Limits Editáveis** | RPM, gap mínimo e concorrência máxima configuráveis |
|
||||
| 🛡 **Proteção de Endpoint API** | Gateway de Auth + bloqueio de provedores para o endpoint `/models` |
|
||||
| 🔒 **Visibilidade de Proxy** | Badges coloridos: 🟢 global, 🟡 provedor, 🔵 por-conexão com exibição de IP |
|
||||
| 🌐 **Proxy em 3 Níveis** | Configure proxies em nível global, por provedor ou por conexão |
|
||||
| Funcionalidade | O que Faz |
|
||||
| ----------------------------------- | ----------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Auto-abertura/fechamento por provedor com limites configuráveis |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semáforo rate-limit para provedores com API key |
|
||||
| 🧠 **Cache Semântico** | Cache de duas camadas (assinatura + semântico) reduz custo e latência |
|
||||
| ⚡ **Idempotência de Requisição** | Janela de dedup de 5s para requisições duplicadas |
|
||||
| 🔒 **Spoofing de Fingerprint TLS** | Bypass de detecção de bot via TLS com wreq-js |
|
||||
| 🌐 **Filtragem de IP** | Allowlist/blocklist para controle de acesso à API |
|
||||
| 📊 **Rate Limits Editáveis** | RPM, gap mínimo e concorrência máxima configuráveis |
|
||||
| 💾 **Persistência de Rate Limits** | Limites aprendidos persistem via SQLite com debounce de 60s + 24h de validade |
|
||||
| 🔄 **Resiliência de Token Refresh** | Circuit breaker por provedor (5 falhas→30min) + timeout de 30s por tentativa |
|
||||
| 🛡 **Proteção de Endpoint API** | Gateway de Auth + bloqueio de provedores para o endpoint `/models` |
|
||||
| 🔒 **Visibilidade de Proxy** | Badges coloridos: 🟢 global, 🟡 provedor, 🔵 por-conexão com exibição de IP |
|
||||
| 🌐 **Proxy em 3 Níveis** | Configure proxies em nível global, por provedor ou por conexão |
|
||||
|
||||
### 📊 Observabilidade e Analytics
|
||||
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Início rápido
|
||||
|
||||
**1. Instale globalmente:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Modelos Personalizados** | Adicione qualquer ID de modelo a qualquer provedor |
|
||||
| 🌐 **Roteador curinga** | Rotear padrões `provider/*` para qualquer provedor dinamicamente |
|
||||
| 🧠 **Pensando no Orçamento** | Modos de passagem, automático, personalizado e adaptativo para modelos de raciocínio |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Injeção imediata do sistema** | Prompt de sistema global aplicado em todas as solicitações |
|
||||
| 📄 **API de respostas** | Suporte completo à API de respostas OpenAI (`/v1/responses`) para Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **Falsificação de impressão digital TLS** | Ignore a detecção de bot baseada em TLS via wreq-js |
|
||||
| 🌐 **Filtragem de IP** | Lista de permissões/lista de bloqueio para controle de acesso à API |
|
||||
| 📊 **Limites de taxas editáveis** | RPM configurável, intervalo mínimo e simultâneo máximo no nível do sistema |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Proteção de endpoint de API** | Autenticação + bloqueio de provedor para o endpoint `/models` |
|
||||
| 🔒 **Visibilidade do proxy** | Crachás codificados por cores: 🟢 global, 🟡 provedor, 🔵 por conexão com display IP |
|
||||
| 🌐 **Configuração de proxy de 3 níveis** | Configurar proxies em nível global, por provedor ou por conexão |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Pornire rapidă
|
||||
|
||||
**1. Instalați la nivel global:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Modele personalizate** | Adăugați orice ID de model oricărui furnizor |
|
||||
| 🌐 **Wildcard Router** | Dirijați dinamic modelele `provider/*` către orice furnizor |
|
||||
| 🧠 **Buget de gândire** | Moduri de trecere, automat, personalizat și adaptiv pentru modelele de raționament |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Injection** | Prompt de sistem global aplicat pentru toate solicitările |
|
||||
| 📄 **Responses API** | Compatibilitate completă cu OpenAI Responses API (`/v1/responses`) pentru Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Ocoliți detectarea botului bazată pe TLS prin wreq-js |
|
||||
| 🌐 **Filtrare IP** | Lista permisă/lista blocată pentru controlul accesului API |
|
||||
| 📊 **Limite de rată editabile** | RPM configurabil, interval minim și concurență maximă la nivel de sistem |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Protecție API Endpoint** | Autentificare + blocare furnizor pentru punctul final `/models` |
|
||||
| 🔒 **Vizibilitatea proxy** | Ecusoane cu coduri de culoare: 🟢 global, 🟡 furnizor, 🔵 per conexiune cu afișaj IP |
|
||||
| 🌐 **Configurare proxy pe 3 niveluri** | Configurați proxy-uri la nivel global, per furnizor sau per conexiune |
|
||||
|
||||
+270
-9
@@ -157,6 +157,263 @@ _Подключайте любую IDE или CLI-инструмент с AI ч
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Быстрый старт
|
||||
|
||||
**1. Установите глобально:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Пользовательские модели** | Добавьте любой ID модели к любому провайдеру |
|
||||
| 🌐 **Wildcard-маршрутизатор** | Маршрутизируйте паттерны `provider/*` к любому провайдеру динамически |
|
||||
| 🧠 **Бюджет рассуждений** | Режимы passthrough, auto, custom и adaptive для моделей рассуждений |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Инъекция System Prompt** | Глобальный system prompt для всех запросов |
|
||||
| 📄 **API Responses** | Полная поддержка OpenAI Responses API (`/v1/responses`) для Codex |
|
||||
|
||||
@@ -327,15 +586,17 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ Устойчивость и безопасность
|
||||
|
||||
| Функция | Что делает |
|
||||
| -------------------------------- | -------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Авто-открытие/закрытие по провайдеру с настраиваемыми порогами |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + семафор для API key провайдеров |
|
||||
| 🧠 **Семантический кеш** | Двухуровневый кеш (сигнатура + семантика) снижает стоимость |
|
||||
| ⚡ **Идемпотентность запросов** | 5с окно дедупликации для дублирующихся запросов |
|
||||
| 🔒 **Спуфинг TLS Fingerprint** | Обход обнаружения ботов через wreq-js |
|
||||
| 🌐 **Фильтрация IP** | Allowlist/blocklist для контроля доступа к API |
|
||||
| 📊 **Настраиваемые Rate Limits** | Настраиваемые RPM, минимальный интервал, макс. конкуррентность |
|
||||
| Функция | Что делает |
|
||||
| -------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **Circuit Breaker** | Авто-открытие/закрытие по провайдеру с настраиваемыми порогами |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + семафор для API key провайдеров |
|
||||
| 🧠 **Семантический кеш** | Двухуровневый кеш (сигнатура + семантика) снижает стоимость |
|
||||
| ⚡ **Идемпотентность запросов** | 5с окно дедупликации для дублирующихся запросов |
|
||||
| 🔒 **Спуфинг TLS Fingerprint** | Обход обнаружения ботов через wreq-js |
|
||||
| 🌐 **Фильтрация IP** | Allowlist/blocklist для контроля доступа к API |
|
||||
| 📊 **Настраиваемые Rate Limits** | Настраиваемые RPM, минимальный интервал, макс. конкуррентность |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
|
||||
### 📊 Наблюдаемость и аналитика
|
||||
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Rýchly štart
|
||||
|
||||
**1. Inštalovať globálne:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Vlastné modely** | Pridajte akékoľvek ID modelu k akémukoľvek poskytovateľovi |
|
||||
| 🌐 **Wildcard Router** | Dynamicky smerujte vzory `provider/*` k akémukoľvek poskytovateľovi |
|
||||
| 🧠 **Premýšľajúci rozpočet** | Priechodný, automatický, vlastný a adaptívny režim pre modely uvažovania |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Promptné vstrekovanie systému** | Globálna systémová výzva aplikovaná na všetky požiadavky |
|
||||
| 📄 **Responses API** | Plná podpora OpenAI Responses API (`/v1/responses`) pre Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 ** Spoofing odtlačkov prstov TLS** | Obíďte detekciu botov na báze TLS cez wreq-js |
|
||||
| 🌐 **Filtrovanie IP** | Zoznam povolených/blokovaných pre riadenie prístupu API |
|
||||
| 📊 **Upraviteľné limity sadzieb** | Konfigurovateľné otáčky za minútu, minimálna medzera a maximálna súbežná rýchlosť na systémovej úrovni |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Auth gating + blokovanie poskytovateľa pre koncový bod `/models` |
|
||||
| 🔒 **Viditeľnosť proxy** | Farebne rozlíšené odznaky: 🟢 globálne, 🟡 poskytovateľ, 🔵 na pripojenie s IP displejom |
|
||||
| 🌐 **3-úrovňová konfigurácia proxy** | Nakonfigurujte proxy na globálnej úrovni, na úrovni jednotlivých poskytovateľov alebo na úrovni pripojenia |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Snabbstart
|
||||
|
||||
**1. Installera globalt:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Anpassade modeller** | Lägg till valfritt modell-ID till valfri leverantör |
|
||||
| 🌐 **Wildcard-router** | Dirigera `provider/*`-mönster till valfri leverantör dynamiskt |
|
||||
| 🧠 **Tänkande budget** | Genomgång, auto, anpassade och adaptiva lägen för resonerande modeller |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt Injection** | Global systemprompt tillämpas på alla förfrågningar |
|
||||
| 📄 **Responses API** | Fullständigt stöd för OpenAI Responses API (`/v1/responses`) för Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Förbi TLS-baserad botdetektering via wreq-js |
|
||||
| 🌐 **IP-filtrering** | Tillåtelselista/blockeringslista för API-åtkomstkontroll |
|
||||
| 📊 **Redigerbara hastighetsgränser** | Konfigurerbart RPM, min gap och max samtidiga på systemnivå |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Auth gating + leverantörsblockering för `/models` slutpunkt |
|
||||
| 🔒 **Proxysynlighet** | Färgkodade märken: 🟢 global, 🟡 leverantör, 🔵 per anslutning med IP-display |
|
||||
| 🌐 **Proxykonfiguration med 3 nivåer** | Konfigurera proxyservrar på global nivå, per leverantör eller per anslutningsnivå |
|
||||
|
||||
+273
-12
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ เริ่มต้นอย่างรวดเร็ว
|
||||
|
||||
**1. ติดตั้งทั่วโลก:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **โมเดลที่กำหนดเอง** | เพิ่ม ID รุ่นใด ๆ ให้กับผู้ให้บริการ |
|
||||
| 🌐 **เราเตอร์ตัวแทน** | กำหนดเส้นทางรูปแบบ `provider/*` ไปยังผู้ให้บริการใดๆ แบบไดนามิก |
|
||||
| 🧠 **คิดงบประมาณ** | โหมดส่งผ่าน, อัตโนมัติ, กำหนดเอง และแบบปรับได้สำหรับโมเดลการให้เหตุผล |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💌 **ระบบพร้อมฉีด** | พร้อมท์ระบบสากลใช้กับคำขอทั้งหมด |
|
||||
| 📄 **API ตอบกลับ** | รองรับ OpenAI Responses API เต็มรูปแบบ (`/v1/responses`) สำหรับ Codex |
|
||||
|
||||
@@ -327,18 +586,20 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ ความยืดหยุ่นและความปลอดภัย
|
||||
|
||||
| คุณสมบัติ | มันทำอะไร |
|
||||
| --------------------------------- | ------------------------------------------------------------------------- |
|
||||
| 🔌 **เซอร์กิตเบรกเกอร์** | เปิด/ปิดอัตโนมัติต่อผู้ให้บริการพร้อมเกณฑ์ที่กำหนดค่าได้ |
|
||||
| 🛡️ **ฝูงต่อต้านฟ้าร้อง** | Mutex + ขีดจำกัดอัตราเซมาฟอร์สำหรับผู้ให้บริการคีย์ API |
|
||||
| 🧠 **แคชความหมาย** | แคชสองชั้น (ลายเซ็น + ความหมาย) ช่วยลดต้นทุน & เวลาแฝง |
|
||||
| ⚡ **ขอ Idempotency** | หน้าต่าง dedup 5s สำหรับคำขอซ้ำ |
|
||||
| 🔒 **การปลอมแปลงลายนิ้วมือ TLS** | เลี่ยงการตรวจจับบอทที่ใช้ TLS ผ่าน wreq-js |
|
||||
| 🌐 **การกรอง IP** | รายการที่อนุญาต/รายการบล็อกสำหรับการควบคุมการเข้าถึง API |
|
||||
| 📊 **ขีดจำกัดอัตราที่แก้ไขได้** | RPM ที่กำหนดค่าได้ ช่องว่างขั้นต่ำ และสูงสุดพร้อมกันที่ระดับระบบ |
|
||||
| 🛡 **การป้องกันปลายทาง API** | การตรวจสอบสิทธิ์ + การบล็อกผู้ให้บริการสำหรับปลายทาง `/models` |
|
||||
| 🔒 **การมองเห็นพร็อกซี** | ป้ายรหัสสี: 🟢 ทั่วโลก 🟡 ผู้ให้บริการ 🔵 ต่อการเชื่อมต่อพร้อมจอแสดงผล IP |
|
||||
| 🌐 **การกำหนดค่าพร็อกซี 3 ระดับ** | กำหนดค่าพร็อกซีในระดับโกลบอล ต่อผู้ให้บริการ หรือต่อการเชื่อมต่อ |
|
||||
| คุณสมบัติ | มันทำอะไร |
|
||||
| --------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **เซอร์กิตเบรกเกอร์** | เปิด/ปิดอัตโนมัติต่อผู้ให้บริการพร้อมเกณฑ์ที่กำหนดค่าได้ |
|
||||
| 🛡️ **ฝูงต่อต้านฟ้าร้อง** | Mutex + ขีดจำกัดอัตราเซมาฟอร์สำหรับผู้ให้บริการคีย์ API |
|
||||
| 🧠 **แคชความหมาย** | แคชสองชั้น (ลายเซ็น + ความหมาย) ช่วยลดต้นทุน & เวลาแฝง |
|
||||
| ⚡ **ขอ Idempotency** | หน้าต่าง dedup 5s สำหรับคำขอซ้ำ |
|
||||
| 🔒 **การปลอมแปลงลายนิ้วมือ TLS** | เลี่ยงการตรวจจับบอทที่ใช้ TLS ผ่าน wreq-js |
|
||||
| 🌐 **การกรอง IP** | รายการที่อนุญาต/รายการบล็อกสำหรับการควบคุมการเข้าถึง API |
|
||||
| 📊 **ขีดจำกัดอัตราที่แก้ไขได้** | RPM ที่กำหนดค่าได้ ช่องว่างขั้นต่ำ และสูงสุดพร้อมกันที่ระดับระบบ |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **การป้องกันปลายทาง API** | การตรวจสอบสิทธิ์ + การบล็อกผู้ให้บริการสำหรับปลายทาง `/models` |
|
||||
| 🔒 **การมองเห็นพร็อกซี** | ป้ายรหัสสี: 🟢 ทั่วโลก 🟡 ผู้ให้บริการ 🔵 ต่อการเชื่อมต่อพร้อมจอแสดงผล IP |
|
||||
| 🌐 **การกำหนดค่าพร็อกซี 3 ระดับ** | กำหนดค่าพร็อกซีในระดับโกลบอล ต่อผู้ให้บริการ หรือต่อการเชื่อมต่อ |
|
||||
|
||||
### 📊 ความสามารถในการสังเกตและการวิเคราะห์
|
||||
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Швидкий старт
|
||||
|
||||
**1. Встановити глобально:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Користувацькі моделі** | Додайте будь-який ідентифікатор моделі до будь-якого постачальника |
|
||||
| 🌐 **Wildcard Router** | Динамічно направляйте шаблони `provider/*` до будь-якого постачальника |
|
||||
| 🧠 **Мислення про бюджет** | Наскрізний, автоматичний, настроюваний і адаптивний режими для моделей міркування |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Швидке впровадження системи** | Глобальне системне підказка застосовується до всіх запитів |
|
||||
| 📄 **API відповідей** | Повна підтримка OpenAI Responses API (`/v1/responses`) для Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **Підробка відбитків пальців TLS** | Обійти виявлення ботів на основі TLS через wreq-js |
|
||||
| 🌐 **IP-фільтрація** | Білий/чорний список для керування доступом API |
|
||||
| 📊 **Редаговані ліміти ставок** | Конфігурація RPM, мінімальний проміжок і максимальна одночасність на рівні системи |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **API Endpoint Protection** | Аутентифікація + блокування постачальника для кінцевої точки `/models` |
|
||||
| 🔒 **Видимість проксі** | Кольорові значки: 🟢 глобальний, 🟡 постачальник, 🔵 кожне підключення з відображенням IP |
|
||||
| 🌐 **3-рівнева конфігурація проксі** | Налаштуйте проксі-сервери на глобальному рівні, на рівні кожного постачальника чи кожного підключення |
|
||||
|
||||
+261
@@ -157,6 +157,263 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ Bắt đầu nhanh
|
||||
|
||||
**1. Cài đặt trên toàn cầu:**
|
||||
@@ -311,6 +568,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🧩 **Mẫu tùy chỉnh** | Thêm bất kỳ ID mẫu nào vào bất kỳ nhà cung cấp nào |
|
||||
| 🌐 **Bộ định tuyến ký tự đại diện** | Định tuyến động các mẫu `provider/*` tới bất kỳ nhà cung cấp nào |
|
||||
| 🧠 **Ngân sách suy nghĩ** | Các chế độ truyền qua, tự động, tùy chỉnh và thích ứng cho các mô hình lý luận |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **Tiêm nhắc nhở hệ thống** | Lời nhắc hệ thống toàn cầu được áp dụng cho tất cả các yêu cầu |
|
||||
| 📄 **API phản hồi** | Hỗ trợ đầy đủ API phản hồi OpenAI (`/v1/responses`) cho Codex |
|
||||
|
||||
@@ -336,6 +595,8 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
| 🔒 **Giả mạo vân tay TLS** | Bỏ qua việc phát hiện bot dựa trên TLS thông qua wreq-js |
|
||||
| 🌐 **Lọc IP** | Danh sách cho phép/danh sách chặn để kiểm soát truy cập API |
|
||||
| 📊 **Giới hạn tỷ lệ có thể chỉnh sửa** | RPM có thể định cấu hình, khoảng cách tối thiểu và đồng thời tối đa ở cấp hệ thống |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
| 🛡 **Bảo vệ điểm cuối API** | Kiểm soát xác thực + chặn nhà cung cấp cho điểm cuối `/models` |
|
||||
| 🔒 **Khả năng hiển thị proxy** | Huy hiệu được mã hóa màu: 🟢 toàn cầu, 🟡 nhà cung cấp, 🔵 mỗi kết nối với màn hình IP |
|
||||
| 🌐 **Cấu hình proxy 3 cấp** | Định cấu hình proxy ở cấp độ toàn cầu, theo nhà cung cấp hoặc theo từng kết nối |
|
||||
|
||||
+283
-22
@@ -157,6 +157,263 @@ _通过 OmniRoute 连接任何 AI 驱动的 IDE 或 CLI 工具 — 免费 API
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
## ⚡ 快速开始
|
||||
|
||||
**1. 全局安装:**
|
||||
@@ -300,19 +557,21 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🧠 路由与智能
|
||||
|
||||
| 功能 | 功能描述 |
|
||||
| ------------------------- | -------------------------------------------------------------------------- |
|
||||
| 🎯 **智能 4 层故障转移** | 自动路由:订阅 → API Key → 低价 → 免费 |
|
||||
| 📊 **实时配额追踪** | 实时 Token 计数 + 每个提供商的重置倒计时 |
|
||||
| 🔄 **格式转换** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro 无缝切换 |
|
||||
| 👥 **多账号支持** | 每个提供商多个账号,智能选择 |
|
||||
| 🔄 **自动令牌刷新** | OAuth 令牌自动刷新并重试 |
|
||||
| 🎨 **自定义组合** | 6 种策略:fill-first、round-robin、p2c、random、least-used、cost-optimized |
|
||||
| 🧩 **自定义模型** | 为任何提供商添加任何模型 ID |
|
||||
| 🌐 **通配符路由** | 动态路由 `provider/*` 模式到任何提供商 |
|
||||
| 🧠 **推理预算** | passthrough、auto、custom 和 adaptive 模式用于推理模型 |
|
||||
| 💬 **System Prompt 注入** | 全局 System Prompt 应用于所有请求 |
|
||||
| 📄 **Responses API** | 完整支持 OpenAI Responses API (`/v1/responses`) 用于 Codex |
|
||||
| 功能 | 功能描述 |
|
||||
| ----------------------------- | ----------------------------------------------------------------------------- |
|
||||
| 🎯 **智能 4 层故障转移** | 自动路由:订阅 → API Key → 低价 → 免费 |
|
||||
| 📊 **实时配额追踪** | 实时 Token 计数 + 每个提供商的重置倒计时 |
|
||||
| 🔄 **格式转换** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro 无缝切换 |
|
||||
| 👥 **多账号支持** | 每个提供商多个账号,智能选择 |
|
||||
| 🔄 **自动令牌刷新** | OAuth 令牌自动刷新并重试 |
|
||||
| 🎨 **自定义组合** | 6 种策略:fill-first、round-robin、p2c、random、least-used、cost-optimized |
|
||||
| 🧩 **自定义模型** | 为任何提供商添加任何模型 ID |
|
||||
| 🌐 **通配符路由** | 动态路由 `provider/*` 模式到任何提供商 |
|
||||
| 🧠 **推理预算** | passthrough、auto、custom 和 adaptive 模式用于推理模型 |
|
||||
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
|
||||
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
|
||||
| 💬 **System Prompt 注入** | 全局 System Prompt 应用于所有请求 |
|
||||
| 📄 **Responses API** | 完整支持 OpenAI Responses API (`/v1/responses`) 用于 Codex |
|
||||
|
||||
### 🎵 多模态 API
|
||||
|
||||
@@ -327,15 +586,17 @@ npm run electron:build:linux # Linux (.AppImage)
|
||||
|
||||
### 🛡️ 弹性与安全
|
||||
|
||||
| 功能 | 功能描述 |
|
||||
| --------------------- | -------------------------------------- |
|
||||
| 🔌 **断路器** | 每个提供商自动打开/关闭,可配置阈值 |
|
||||
| 🛡️ **反惊群** | Mutex + 信号量限速用于 API Key 提供商 |
|
||||
| 🧠 **语义缓存** | 两层缓存(签名 + 语义)降低成本和延迟 |
|
||||
| ⚡ **请求幂等性** | 5 秒去重窗口防止重复请求 |
|
||||
| 🔒 **TLS 指纹伪装** | 通过 wreq-js 绕过基于 TLS 的机器人检测 |
|
||||
| 🌐 **IP 过滤** | 白名单/黑名单用于 API 访问控制 |
|
||||
| 📊 **可编辑速率限制** | 可配置的 RPM、最小间隔和最大并发 |
|
||||
| 功能 | 功能描述 |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| 🔌 **断路器** | 每个提供商自动打开/关闭,可配置阈值 |
|
||||
| 🛡️ **反惊群** | Mutex + 信号量限速用于 API Key 提供商 |
|
||||
| 🧠 **语义缓存** | 两层缓存(签名 + 语义)降低成本和延迟 |
|
||||
| ⚡ **请求幂等性** | 5 秒去重窗口防止重复请求 |
|
||||
| 🔒 **TLS 指纹伪装** | 通过 wreq-js 绕过基于 TLS 的机器人检测 |
|
||||
| 🌐 **IP 过滤** | 白名单/黑名单用于 API 访问控制 |
|
||||
| 📊 **可编辑速率限制** | 可配置的 RPM、最小间隔和最大并发 |
|
||||
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
|
||||
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
|
||||
|
||||
### 📊 可观察性与分析
|
||||
|
||||
|
||||
+1
-1
@@ -48,7 +48,7 @@ Four modes for debugging API translations: **Playground** (format converter), **
|
||||
|
||||
## ⚙️ Settings
|
||||
|
||||
General settings, system storage, backup management (export/import database), appearance (dark/light mode), security (includes API endpoint protection and custom provider blocking), routing, resilience, and advanced configuration.
|
||||
General settings, system storage, backup management (export/import database), appearance (dark/light mode), security (includes API endpoint protection and custom provider blocking), routing (model aliases, background task degradation), resilience (rate limit persistence), and advanced configuration.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -210,6 +210,41 @@ When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex
|
||||
|
||||
---
|
||||
|
||||
## Optional RAG / LLM failure taxonomy (16 problems)
|
||||
|
||||
Some OmniRoute users place the gateway in front of RAG or agent stacks. In those setups it is common to see a strange pattern: OmniRoute looks healthy (providers up, routing profiles ok, no rate limit alerts) but the final answer is still wrong.
|
||||
|
||||
In practice these incidents usually come from the downstream RAG pipeline, not from the gateway itself.
|
||||
|
||||
If you want a shared vocabulary to describe those failures you can use the WFGY ProblemMap, an external MIT license text resource that defines sixteen recurring RAG / LLM failure patterns. At a high level it covers:
|
||||
|
||||
- retrieval drift and broken context boundaries
|
||||
- empty or stale indexes and vector stores
|
||||
- embedding versus semantic mismatch
|
||||
- prompt assembly and context window issues
|
||||
- logic collapse and overconfident answers
|
||||
- long chain and agent coordination failures
|
||||
- multi agent memory and role drift
|
||||
- deployment and bootstrap ordering problems
|
||||
|
||||
The idea is simple:
|
||||
|
||||
1. When you investigate a bad response, capture:
|
||||
- user task and request
|
||||
- route or provider combo in OmniRoute
|
||||
- any RAG context used downstream (retrieved documents, tool calls, etc)
|
||||
2. Map the incident to one or two WFGY ProblemMap numbers (`No.1` … `No.16`).
|
||||
3. Store the number in your own dashboard, runbook, or incident tracker next to the OmniRoute logs.
|
||||
4. Use the corresponding WFGY page to decide whether you need to change your RAG stack, retriever, or routing strategy.
|
||||
|
||||
Full text and concrete recipes live here (MIT license, text only):
|
||||
|
||||
[WFGY ProblemMap README](https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md)
|
||||
|
||||
You can ignore this section if you do not run RAG or agent pipelines behind OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## Still Stuck?
|
||||
|
||||
- **GitHub Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
|
||||
|
||||
@@ -16,7 +16,7 @@ Administrer AI-udbyderforbindelser: OAuth-udbydere (Claude Code, Codex, Gemini C
|
||||
|
||||
## 🎨 Kombinationer
|
||||
|
||||
Opret modelrouting-kombinationer med 6 strategier: Fyld-først, round-robin, power-of-to-choices, tilfældig, mindst brugt og omkostningsoptimeret. Hver combo kæder flere modeller med automatisk fallback.
|
||||
Opret modelrouting (model aliases, background task degradation)-kombinationer med 6 strategier: Fyld-først, round-robin, power-of-to-choices, tilfældig, mindst brugt og omkostningsoptimeret. Hver combo kæder flere modeller med automatisk fallback.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ Gestisci le connessioni dei provider AI: provider OAuth (Claude Code, Codex, Gem
|
||||
|
||||
## 🎨Combo
|
||||
|
||||
Crea combinazioni di routing del modello con 6 strategie: riempimento prima, round robin, scelta potenza di due, casuale, meno utilizzata e con ottimizzazione dei costi. Ogni combo concatena più modelli con fallback automatico.
|
||||
Crea combinazioni di routing (model aliases, background task degradation) del modello con 6 strategie: riempimento prima, round robin, scelta potenza di due, casuale, meno utilizzata e con ottimizzazione dei costi. Ogni combo concatena più modelli con fallback automatico.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ Vier modi voor het debuggen van API-vertalingen: **Playground** (formaatconverte
|
||||
|
||||
## ⚙️ Instellingen
|
||||
|
||||
Algemene instellingen, systeemopslag, back-upbeheer (database exporteren/importeren), uiterlijk (donker/licht-modus), beveiliging (inclusief API-eindpuntbescherming en aangepaste providerblokkering), routing, veerkracht en geavanceerde configuratie.
|
||||
Algemene instellingen, systeemopslag, back-upbeheer (database exporteren/importeren), uiterlijk (donker/licht-modus), beveiliging (inclusief API-eindpuntbescherming en aangepaste providerblokkering), routing (model aliases, background task degradation), veerkracht en geavanceerde configuratie.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ Apat na mode para sa pag-debug ng mga pagsasalin ng API: **Playground** (format
|
||||
|
||||
## ⚙️ Mga Setting
|
||||
|
||||
Mga pangkalahatang setting, system storage, backup management (export/import database), hitsura (dark/light mode), seguridad (kasama ang API endpoint protection at custom provider blocking), routing, resilience, at advanced configuration.
|
||||
Mga pangkalahatang setting, system storage, backup management (export/import database), hitsura (dark/light mode), seguridad (kasama ang API endpoint protection at custom provider blocking), routing (model aliases, background task degradation), resilience, at advanced configuration.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ Zarządzaj połączeniami dostawców AI: dostawcy OAuth (Claude Code, Codex, Gem
|
||||
|
||||
## 🎨 Kombinacje
|
||||
|
||||
Twórz kombinacje routingu modeli za pomocą 6 strategii: najpierw wypełnij, okrężnie, siła dwóch wyborów, losowa, najrzadziej używana i zoptymalizowana pod względem kosztów. Każda kombinacja łączy wiele modeli z automatycznym cofaniem.
|
||||
Twórz kombinacje routing (model aliases, background task degradation)u modeli za pomocą 6 strategii: najpierw wypełnij, okrężnie, siła dwóch wyborów, losowa, najrzadziej używana i zoptymalizowana pod względem kosztów. Każda kombinacja łączy wiele modeli z automatycznym cofaniem.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ Hantera AI-leverantörsanslutningar: OAuth-leverantörer (Claude Code, Codex, Ge
|
||||
|
||||
## 🎨 Combos
|
||||
|
||||
Skapa modell routing-kombinationer med 6 strategier: fyll först, round-robin, kraft-av-två-val, slumpmässig, minst använda och kostnadsoptimerad. Varje combo kedjer flera modeller med automatisk reserv.
|
||||
Skapa modell routing (model aliases, background task degradation)-kombinationer med 6 strategier: fyll först, round-robin, kraft-av-två-val, slumpmässig, minst använda och kostnadsoptimerad. Varje combo kedjer flera modeller med automatisk reserv.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute-desktop",
|
||||
"version": "1.6.8",
|
||||
"version": "1.6.9",
|
||||
"description": "OmniRoute Desktop Application",
|
||||
"main": "main.js",
|
||||
"author": "OmniRoute Team",
|
||||
|
||||
@@ -70,6 +70,40 @@ export const AUDIO_TRANSCRIPTION_PROVIDERS: Record<string, AudioProvider> = {
|
||||
{ id: "universal-2", name: "Universal 2" },
|
||||
],
|
||||
},
|
||||
|
||||
nvidia: {
|
||||
id: "nvidia",
|
||||
baseUrl: "https://integrate.api.nvidia.com/v1/audio/transcriptions",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
format: "nvidia-asr",
|
||||
models: [
|
||||
{ id: "nvidia/parakeet-ctc-1.1b-asr", name: "Parakeet CTC 1.1B" },
|
||||
],
|
||||
},
|
||||
|
||||
huggingface: {
|
||||
id: "huggingface",
|
||||
baseUrl: "https://api-inference.huggingface.co/models",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
format: "huggingface-asr",
|
||||
models: [
|
||||
{ id: "openai/whisper-large-v3", name: "Whisper Large v3 (HF)" },
|
||||
{ id: "openai/whisper-small", name: "Whisper Small (HF)" },
|
||||
],
|
||||
},
|
||||
|
||||
qwen: {
|
||||
id: "qwen",
|
||||
baseUrl: "http://localhost:8000/v1/audio/transcriptions",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "openai",
|
||||
models: [
|
||||
{ id: "qwen3-asr", name: "Qwen3 ASR" },
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
export const AUDIO_SPEECH_PROVIDERS: Record<string, AudioProvider> = {
|
||||
@@ -106,6 +140,75 @@ export const AUDIO_SPEECH_PROVIDERS: Record<string, AudioProvider> = {
|
||||
{ id: "aura-stella-en", name: "Aura Stella (EN)" },
|
||||
],
|
||||
},
|
||||
|
||||
nvidia: {
|
||||
id: "nvidia",
|
||||
baseUrl: "https://integrate.api.nvidia.com/v1/audio/speech",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
format: "nvidia-tts",
|
||||
models: [
|
||||
{ id: "nvidia/fastpitch", name: "FastPitch" },
|
||||
{ id: "nvidia/tacotron2", name: "Tacotron2" },
|
||||
],
|
||||
},
|
||||
|
||||
elevenlabs: {
|
||||
id: "elevenlabs",
|
||||
baseUrl: "https://api.elevenlabs.io/v1/text-to-speech",
|
||||
authType: "apikey",
|
||||
authHeader: "xi-api-key",
|
||||
format: "elevenlabs",
|
||||
models: [
|
||||
{ id: "eleven_multilingual_v2", name: "Eleven Multilingual v2" },
|
||||
{ id: "eleven_turbo_v2_5", name: "Eleven Turbo v2.5" },
|
||||
],
|
||||
},
|
||||
|
||||
huggingface: {
|
||||
id: "huggingface",
|
||||
baseUrl: "https://api-inference.huggingface.co/models",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
format: "huggingface-tts",
|
||||
models: [
|
||||
{ id: "facebook/mms-tts-eng", name: "MMS TTS English" },
|
||||
{ id: "microsoft/speecht5_tts", name: "SpeechT5 TTS" },
|
||||
],
|
||||
},
|
||||
|
||||
coqui: {
|
||||
id: "coqui",
|
||||
baseUrl: "http://localhost:5002/api/tts",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "coqui",
|
||||
models: [
|
||||
{ id: "tts_models/en/ljspeech/tacotron2-DDC", name: "Tacotron2 DDC (LJSpeech)" },
|
||||
],
|
||||
},
|
||||
|
||||
tortoise: {
|
||||
id: "tortoise",
|
||||
baseUrl: "http://localhost:5000/api/tts",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "tortoise",
|
||||
models: [
|
||||
{ id: "tortoise-v2", name: "Tortoise v2" },
|
||||
],
|
||||
},
|
||||
|
||||
qwen: {
|
||||
id: "qwen",
|
||||
baseUrl: "http://localhost:8000/v1/audio/speech",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "openai",
|
||||
models: [
|
||||
{ id: "qwen3-tts", name: "Qwen3 TTS" },
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -108,6 +108,32 @@ export const IMAGE_PROVIDERS = {
|
||||
],
|
||||
supportedSizes: ["1024x1024"],
|
||||
},
|
||||
|
||||
sdwebui: {
|
||||
id: "sdwebui",
|
||||
baseUrl: "http://localhost:7860/sdapi/v1/txt2img",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "sdwebui",
|
||||
models: [
|
||||
{ id: "stable-diffusion-v1-5", name: "Stable Diffusion v1.5" },
|
||||
{ id: "sdxl-base-1.0", name: "SDXL Base 1.0" },
|
||||
],
|
||||
supportedSizes: ["512x512", "768x768", "1024x1024"],
|
||||
},
|
||||
|
||||
comfyui: {
|
||||
id: "comfyui",
|
||||
baseUrl: "http://localhost:8188",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "comfyui",
|
||||
models: [
|
||||
{ id: "flux-dev", name: "FLUX Dev" },
|
||||
{ id: "sdxl", name: "SDXL" },
|
||||
],
|
||||
supportedSizes: ["512x512", "768x768", "1024x1024"],
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* Music Generation Provider Registry
|
||||
*
|
||||
* Defines providers that support the /v1/music/generations endpoint.
|
||||
* Currently supports local providers (ComfyUI with audio models).
|
||||
*/
|
||||
|
||||
import { parseModelFromRegistry, getAllModelsFromRegistry } from "./registryUtils.ts";
|
||||
|
||||
interface MusicModel {
|
||||
id: string;
|
||||
name: string;
|
||||
}
|
||||
|
||||
interface MusicProvider {
|
||||
id: string;
|
||||
baseUrl: string;
|
||||
authType: string;
|
||||
authHeader: string;
|
||||
format: string;
|
||||
models: MusicModel[];
|
||||
}
|
||||
|
||||
export const MUSIC_PROVIDERS: Record<string, MusicProvider> = {
|
||||
comfyui: {
|
||||
id: "comfyui",
|
||||
baseUrl: "http://localhost:8188",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "comfyui",
|
||||
models: [
|
||||
{ id: "stable-audio-open", name: "Stable Audio Open" },
|
||||
{ id: "musicgen-medium", name: "MusicGen Medium" },
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Get music provider config by ID
|
||||
*/
|
||||
export function getMusicProvider(providerId: string): MusicProvider | null {
|
||||
return MUSIC_PROVIDERS[providerId] || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse music model string (format: "provider/model" or just "model")
|
||||
*/
|
||||
export function parseMusicModel(modelStr: string | null) {
|
||||
return parseModelFromRegistry(modelStr, MUSIC_PROVIDERS);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all music models as a flat list
|
||||
*/
|
||||
export function getAllMusicModels() {
|
||||
return getAllModelsFromRegistry(MUSIC_PROVIDERS);
|
||||
}
|
||||
@@ -111,7 +111,10 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
clientSecretDefault: "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",
|
||||
},
|
||||
models: [
|
||||
{ id: "gemini-3-pro-preview", name: "Gemini 3 Pro Preview" },
|
||||
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
|
||||
{ id: "gemini-3.1-flash", name: "Gemini 3.1 Flash" },
|
||||
{ id: "gemini-3-pro-preview", name: "Gemini 3.0 Pro Preview" },
|
||||
{ id: "gemini-3-flash-preview", name: "Gemini 3.0 Flash Preview" },
|
||||
{ id: "gemini-2.5-pro", name: "Gemini 2.5 Pro" },
|
||||
{ id: "gemini-2.5-flash", name: "Gemini 2.5 Flash" },
|
||||
{ id: "gemini-2.5-flash-lite", name: "Gemini 2.5 Flash Lite" },
|
||||
@@ -137,8 +140,10 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
clientSecretDefault: "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",
|
||||
},
|
||||
models: [
|
||||
{ id: "gemini-3-flash-preview", name: "Gemini 3 Flash Preview" },
|
||||
{ id: "gemini-3-pro-preview", name: "Gemini 3 Pro Preview" },
|
||||
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
|
||||
{ id: "gemini-3.1-flash", name: "Gemini 3.1 Flash" },
|
||||
{ id: "gemini-3-flash-preview", name: "Gemini 3.0 Flash Preview" },
|
||||
{ id: "gemini-3-pro-preview", name: "Gemini 3.0 Pro Preview" },
|
||||
{ id: "gemini-2.5-pro", name: "Gemini 2.5 Pro" },
|
||||
{ id: "gemini-2.5-flash", name: "Gemini 2.5 Flash" },
|
||||
{ id: "gemini-2.5-flash-lite", name: "Gemini 2.5 Flash Lite" },
|
||||
@@ -268,7 +273,8 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
{ id: "claude-sonnet-4-6", name: "Claude Sonnet 4.6" },
|
||||
{ id: "gemini-3.1-pro-high", name: "Gemini 3.1 Pro High" },
|
||||
{ id: "gemini-3.1-pro-low", name: "Gemini 3.1 Pro Low" },
|
||||
{ id: "gemini-3-flash", name: "Gemini 3 Flash" },
|
||||
{ id: "gemini-3.1-flash", name: "Gemini 3.1 Flash" },
|
||||
{ id: "gemini-3-flash", name: "Gemini 3.0 Flash" },
|
||||
{ id: "gpt-oss-120b-medium", name: "GPT OSS 120B Medium" },
|
||||
],
|
||||
},
|
||||
@@ -635,6 +641,25 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
],
|
||||
},
|
||||
|
||||
blackbox: {
|
||||
id: "blackbox",
|
||||
alias: "bb",
|
||||
format: "openai",
|
||||
executor: "default",
|
||||
baseUrl: "https://api.blackbox.ai/v1/chat/completions",
|
||||
modelsUrl: "https://api.blackbox.ai/v1/models",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
models: [
|
||||
{ id: "gpt-4o", name: "GPT-4o" },
|
||||
{ id: "gemini-2.5-flash", name: "Gemini 2.5 Flash" },
|
||||
{ id: "claude-sonnet-4", name: "Claude Sonnet 4" },
|
||||
{ id: "deepseek-v3", name: "DeepSeek V3" },
|
||||
{ id: "blackboxai", name: "Blackbox AI" },
|
||||
{ id: "blackboxai-pro", name: "Blackbox AI Pro" },
|
||||
],
|
||||
},
|
||||
|
||||
xai: {
|
||||
id: "xai",
|
||||
alias: "xai",
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* Shared Registry Utilities
|
||||
*
|
||||
* Common interfaces and helpers used by all provider registries
|
||||
* (audio, image, video, music). Extracts duplicated patterns into
|
||||
* reusable functions.
|
||||
*/
|
||||
|
||||
export interface BaseModel {
|
||||
id: string;
|
||||
name: string;
|
||||
}
|
||||
|
||||
export interface BaseProvider<M extends BaseModel = BaseModel> {
|
||||
id: string;
|
||||
baseUrl: string;
|
||||
authType: string; // "apikey" | "oauth" | "none"
|
||||
authHeader: string; // "bearer" | "token" | "xi-api-key" | "x-api-key" | "none"
|
||||
format?: string;
|
||||
models: M[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a "provider/model" string against a registry.
|
||||
* Supports both "provider/model" prefix and bare "model" lookup.
|
||||
*/
|
||||
export function parseModelFromRegistry<P extends BaseProvider>(
|
||||
modelStr: string | null,
|
||||
registry: Record<string, P>
|
||||
): { provider: string | null; model: string | null } {
|
||||
if (!modelStr) return { provider: null, model: null };
|
||||
|
||||
// Try each provider prefix
|
||||
for (const [providerId] of Object.entries(registry)) {
|
||||
if (modelStr.startsWith(providerId + "/")) {
|
||||
return { provider: providerId, model: modelStr.slice(providerId.length + 1) };
|
||||
}
|
||||
}
|
||||
|
||||
// No provider prefix — try to find the model in any provider
|
||||
for (const [providerId, config] of Object.entries(registry)) {
|
||||
if (config.models.some((m) => m.id === modelStr)) {
|
||||
return { provider: providerId, model: modelStr };
|
||||
}
|
||||
}
|
||||
|
||||
return { provider: null, model: modelStr };
|
||||
}
|
||||
|
||||
/**
|
||||
* Flatten all models from a registry into a list with provider info.
|
||||
* Optionally merge extra fields per provider via the `extra` callback.
|
||||
*/
|
||||
export function getAllModelsFromRegistry<P extends BaseProvider>(
|
||||
registry: Record<string, P>,
|
||||
extra?: (providerId: string, config: P) => Record<string, unknown>
|
||||
): Array<{ id: string; name: string; provider: string } & Record<string, unknown>> {
|
||||
const models: Array<{ id: string; name: string; provider: string } & Record<string, unknown>> =
|
||||
[];
|
||||
|
||||
for (const [providerId, config] of Object.entries(registry)) {
|
||||
const extraFields = extra ? extra(providerId, config) : {};
|
||||
for (const model of config.models) {
|
||||
models.push({
|
||||
id: `${providerId}/${model.id}`,
|
||||
name: model.name,
|
||||
provider: providerId,
|
||||
...extraFields,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return models;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build auth headers for a provider.
|
||||
* Handles bearer, token, xi-api-key, x-api-key, and none.
|
||||
*/
|
||||
export function buildAuthHeaders(
|
||||
provider: BaseProvider,
|
||||
token: string | null
|
||||
): Record<string, string> {
|
||||
if (provider.authType === "none" || provider.authHeader === "none" || !token) {
|
||||
return {};
|
||||
}
|
||||
|
||||
switch (provider.authHeader) {
|
||||
case "token":
|
||||
return { Authorization: `Token ${token}` };
|
||||
case "xi-api-key":
|
||||
return { "xi-api-key": token };
|
||||
case "x-api-key":
|
||||
return { "x-api-key": token };
|
||||
case "bearer":
|
||||
default:
|
||||
return { Authorization: `Bearer ${token}` };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* Video Generation Provider Registry
|
||||
*
|
||||
* Defines providers that support the /v1/videos/generations endpoint.
|
||||
* Currently supports local providers (ComfyUI, SD WebUI with AnimateDiff).
|
||||
*/
|
||||
|
||||
import { parseModelFromRegistry, getAllModelsFromRegistry } from "./registryUtils.ts";
|
||||
|
||||
interface VideoModel {
|
||||
id: string;
|
||||
name: string;
|
||||
}
|
||||
|
||||
interface VideoProvider {
|
||||
id: string;
|
||||
baseUrl: string;
|
||||
authType: string;
|
||||
authHeader: string;
|
||||
format: string;
|
||||
models: VideoModel[];
|
||||
}
|
||||
|
||||
export const VIDEO_PROVIDERS: Record<string, VideoProvider> = {
|
||||
comfyui: {
|
||||
id: "comfyui",
|
||||
baseUrl: "http://localhost:8188",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "comfyui",
|
||||
models: [
|
||||
{ id: "animatediff", name: "AnimateDiff" },
|
||||
{ id: "svd-xt", name: "Stable Video Diffusion XT" },
|
||||
],
|
||||
},
|
||||
|
||||
sdwebui: {
|
||||
id: "sdwebui",
|
||||
baseUrl: "http://localhost:7860",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
format: "sdwebui-video",
|
||||
models: [
|
||||
{ id: "animatediff-webui", name: "AnimateDiff (WebUI)" },
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Get video provider config by ID
|
||||
*/
|
||||
export function getVideoProvider(providerId: string): VideoProvider | null {
|
||||
return VIDEO_PROVIDERS[providerId] || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse video model string (format: "provider/model" or just "model")
|
||||
*/
|
||||
export function parseVideoModel(modelStr: string | null) {
|
||||
return parseModelFromRegistry(modelStr, VIDEO_PROVIDERS);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all video models as a flat list
|
||||
*/
|
||||
export function getAllVideoModels() {
|
||||
return getAllModelsFromRegistry(VIDEO_PROVIDERS);
|
||||
}
|
||||
@@ -36,7 +36,18 @@ export class AntigravityExecutor extends BaseExecutor {
|
||||
}
|
||||
|
||||
transformRequest(model, body, stream, credentials) {
|
||||
const projectId = credentials?.projectId || this.generateProjectId();
|
||||
const bodyProjectId = body?.project;
|
||||
const credentialsProjectId = credentials?.projectId;
|
||||
const hasExplicitProject = !!(bodyProjectId || credentialsProjectId);
|
||||
const projectId = bodyProjectId || credentialsProjectId || this.generateProjectId();
|
||||
|
||||
if (!hasExplicitProject) {
|
||||
console.warn(
|
||||
`[Antigravity] ⚠️ No projectId provided via body or credentials — using generated fallback "${projectId}". ` +
|
||||
`This may cause 404 errors if the account has no active GCP project. ` +
|
||||
`Ensure the OAuth token includes a valid project or the request includes a project field.`
|
||||
);
|
||||
}
|
||||
|
||||
// Fix contents for Claude models via Antigravity
|
||||
const normalizedContents =
|
||||
|
||||
@@ -43,6 +43,19 @@ export class BaseExecutor {
|
||||
...this.config.headers,
|
||||
};
|
||||
|
||||
// Allow per-provider User-Agent override via environment variable.
|
||||
// Example: CLAUDE_USER_AGENT="my-agent/2.0" overrides the default for the Claude provider.
|
||||
const providerId = this.config?.id || this.provider;
|
||||
if (providerId) {
|
||||
const envKey = `${providerId.toUpperCase().replace(/[^A-Z0-9]/g, "_")}_USER_AGENT`;
|
||||
const envUA = process.env[envKey]?.trim();
|
||||
if (envUA) {
|
||||
// Override both common casing variants
|
||||
headers["User-Agent"] = envUA;
|
||||
if (headers["user-agent"]) headers["user-agent"] = envUA;
|
||||
}
|
||||
}
|
||||
|
||||
if (credentials.accessToken) {
|
||||
headers["Authorization"] = `Bearer ${credentials.accessToken}`;
|
||||
} else if (credentials.apiKey) {
|
||||
|
||||
@@ -6,22 +6,54 @@ import { getCorsOrigin } from "../utils/cors.ts";
|
||||
* Returns audio binary stream.
|
||||
*
|
||||
* Supported provider formats:
|
||||
* - OpenAI: standard JSON → audio stream proxy
|
||||
* - OpenAI / Qwen3 (openai-compatible): standard JSON → audio stream proxy
|
||||
* - Hyperbolic: POST { text } → { audio: base64 }
|
||||
* - Deepgram: POST { text } with model via query param, Token auth
|
||||
* - ElevenLabs: POST { text, model_id } to /v1/text-to-speech/{voice_id}
|
||||
* - Nvidia NIM: POST { input: { text }, voice, model } → audio binary
|
||||
* - HuggingFace Inference: POST { inputs: text } to /models/{model_id}
|
||||
* - Coqui TTS: POST { text, speaker_id } → WAV audio (local, no auth)
|
||||
* - Tortoise TTS: POST { text, voice } → audio binary (local, no auth)
|
||||
*/
|
||||
|
||||
import { getSpeechProvider, parseSpeechModel } from "../config/audioRegistry.ts";
|
||||
import { buildAuthHeaders } from "../config/registryUtils.ts";
|
||||
import { errorResponse } from "../utils/error.ts";
|
||||
|
||||
/**
|
||||
* Build auth header for a speech provider
|
||||
* Return a CORS error response from an upstream fetch failure
|
||||
*/
|
||||
function buildAuthHeader(providerConfig, token) {
|
||||
if (providerConfig.authHeader === "token") {
|
||||
return { Authorization: `Token ${token}` };
|
||||
}
|
||||
return { Authorization: `Bearer ${token}` };
|
||||
function upstreamErrorResponse(res, errText) {
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Return a CORS audio stream response
|
||||
*/
|
||||
function audioStreamResponse(res, defaultContentType = "audio/mpeg") {
|
||||
const contentType = res.headers.get("content-type") || defaultContentType;
|
||||
return new Response(res.body, {
|
||||
status: 200,
|
||||
headers: {
|
||||
"Content-Type": contentType,
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
"Transfer-Encoding": "chunked",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a path segment to prevent path traversal / SSRF.
|
||||
* Returns true if safe, false if it contains traversal sequences.
|
||||
*/
|
||||
function isValidPathSegment(segment: string): boolean {
|
||||
return !segment.includes("..") && !segment.includes("//");
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -32,20 +64,13 @@ async function handleHyperbolicSpeech(providerConfig, body, token) {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...buildAuthHeader(providerConfig, token),
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
},
|
||||
body: JSON.stringify({ text: body.input }),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
@@ -72,29 +97,153 @@ async function handleDeepgramSpeech(providerConfig, body, modelId, token) {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...buildAuthHeader(providerConfig, token),
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
},
|
||||
body: JSON.stringify({ text: body.input }),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const contentType = res.headers.get("content-type") || "audio/mpeg";
|
||||
return audioStreamResponse(res);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle ElevenLabs TTS
|
||||
* POST {baseUrl}/{voice_id} with { text, model_id }
|
||||
* voice_id is mapped from the OpenAI `voice` parameter
|
||||
*/
|
||||
async function handleElevenLabsSpeech(providerConfig, body, modelId, token) {
|
||||
// ElevenLabs uses voice_id in URL path; default to "21m00Tcm4TlvDq8ikWAM" (Rachel)
|
||||
const voiceId = body.voice || "21m00Tcm4TlvDq8ikWAM";
|
||||
if (!isValidPathSegment(voiceId)) {
|
||||
return errorResponse(400, "Invalid voice ID");
|
||||
}
|
||||
const url = `${providerConfig.baseUrl}/${voiceId}`;
|
||||
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
},
|
||||
body: JSON.stringify({
|
||||
text: body.input,
|
||||
model_id: modelId,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
return audioStreamResponse(res);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle Nvidia NIM TTS
|
||||
* POST with { input: { text }, voice, model } → audio binary
|
||||
*/
|
||||
async function handleNvidiaTtsSpeech(providerConfig, body, modelId, token) {
|
||||
const res = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
},
|
||||
body: JSON.stringify({
|
||||
input: { text: body.input },
|
||||
voice: body.voice || "default",
|
||||
model: modelId,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
return audioStreamResponse(res, "audio/wav");
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle HuggingFace Inference TTS
|
||||
* POST {baseUrl}/{model_id} with { inputs: text } → audio binary
|
||||
*/
|
||||
async function handleHuggingFaceTtsSpeech(providerConfig, body, modelId, token) {
|
||||
if (!isValidPathSegment(modelId)) {
|
||||
return errorResponse(400, "Invalid model ID");
|
||||
}
|
||||
const url = `${providerConfig.baseUrl}/${modelId}`;
|
||||
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
},
|
||||
body: JSON.stringify({ inputs: body.input }),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
return audioStreamResponse(res, "audio/wav");
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle Coqui TTS (local, no auth)
|
||||
* POST {baseUrl} with { text, speaker_id } → WAV audio
|
||||
*/
|
||||
async function handleCoquiSpeech(providerConfig, body) {
|
||||
const res = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
text: body.input,
|
||||
speaker_id: body.voice || undefined,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const contentType = res.headers.get("content-type") || "audio/wav";
|
||||
return new Response(res.body, {
|
||||
status: 200,
|
||||
headers: {
|
||||
"Content-Type": contentType,
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle Tortoise TTS (local, no auth)
|
||||
* POST {baseUrl} with { text, voice } → audio binary
|
||||
*/
|
||||
async function handleTortoiseSpeech(providerConfig, body) {
|
||||
const res = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
text: body.input,
|
||||
voice: body.voice || "random",
|
||||
}),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const contentType = res.headers.get("content-type") || "audio/wav";
|
||||
return new Response(res.body, {
|
||||
status: 200,
|
||||
headers: {
|
||||
"Content-Type": contentType,
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
"Transfer-Encoding": "chunked",
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -122,12 +271,13 @@ export async function handleAudioSpeech({ body, credentials }) {
|
||||
if (!providerConfig) {
|
||||
return errorResponse(
|
||||
400,
|
||||
`No speech provider found for model "${body.model}". Available: openai, hyperbolic, deepgram`
|
||||
`No speech provider found for model "${body.model}". Available: openai, hyperbolic, deepgram, nvidia, elevenlabs, huggingface, coqui, tortoise, qwen`
|
||||
);
|
||||
}
|
||||
|
||||
const token = credentials?.apiKey || credentials?.accessToken;
|
||||
if (!token) {
|
||||
// Skip credential check for local providers (authType: "none")
|
||||
const token = providerConfig.authType === "none" ? null : (credentials?.apiKey || credentials?.accessToken);
|
||||
if (providerConfig.authType !== "none" && !token) {
|
||||
return errorResponse(401, `No credentials for speech provider: ${providerId}`);
|
||||
}
|
||||
|
||||
@@ -141,12 +291,32 @@ export async function handleAudioSpeech({ body, credentials }) {
|
||||
return handleDeepgramSpeech(providerConfig, body, modelId, token);
|
||||
}
|
||||
|
||||
// Default: OpenAI-compatible JSON → audio stream proxy
|
||||
if (providerConfig.format === "elevenlabs") {
|
||||
return handleElevenLabsSpeech(providerConfig, body, modelId, token);
|
||||
}
|
||||
|
||||
if (providerConfig.format === "nvidia-tts") {
|
||||
return handleNvidiaTtsSpeech(providerConfig, body, modelId, token);
|
||||
}
|
||||
|
||||
if (providerConfig.format === "huggingface-tts") {
|
||||
return handleHuggingFaceTtsSpeech(providerConfig, body, modelId, token);
|
||||
}
|
||||
|
||||
if (providerConfig.format === "coqui") {
|
||||
return handleCoquiSpeech(providerConfig, body);
|
||||
}
|
||||
|
||||
if (providerConfig.format === "tortoise") {
|
||||
return handleTortoiseSpeech(providerConfig, body);
|
||||
}
|
||||
|
||||
// Default: OpenAI-compatible JSON → audio stream proxy (also used by Qwen3)
|
||||
const res = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...buildAuthHeader(providerConfig, token),
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: modelId,
|
||||
@@ -158,27 +328,11 @@ export async function handleAudioSpeech({ body, credentials }) {
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
// Stream audio response back to client
|
||||
const contentType = res.headers.get("content-type") || "audio/mpeg";
|
||||
return new Response(res.body, {
|
||||
status: 200,
|
||||
headers: {
|
||||
"Content-Type": contentType,
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
"Transfer-Encoding": "chunked",
|
||||
},
|
||||
});
|
||||
return audioStreamResponse(res);
|
||||
} catch (err) {
|
||||
return errorResponse(500, `Speech request failed: ${err.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,22 +6,35 @@ import { getCorsOrigin } from "../utils/cors.ts";
|
||||
* Proxies multipart/form-data to upstream providers.
|
||||
*
|
||||
* Supported provider formats:
|
||||
* - OpenAI/Groq: standard multipart form-data proxy
|
||||
* - OpenAI/Groq/Qwen3: standard multipart form-data proxy
|
||||
* - Deepgram: raw binary audio POST with model via query param
|
||||
* - AssemblyAI: async workflow (upload → submit → poll)
|
||||
* - Nvidia NIM: multipart POST, transform response to { text }
|
||||
* - HuggingFace Inference: POST raw binary to /models/{model_id}
|
||||
*/
|
||||
|
||||
import { getTranscriptionProvider, parseTranscriptionModel } from "../config/audioRegistry.ts";
|
||||
import { buildAuthHeaders } from "../config/registryUtils.ts";
|
||||
import { errorResponse } from "../utils/error.ts";
|
||||
|
||||
/**
|
||||
* Build auth header for a transcription provider
|
||||
* Return a CORS error response from an upstream fetch failure
|
||||
*/
|
||||
function buildAuthHeader(providerConfig, token) {
|
||||
if (providerConfig.authHeader === "token") {
|
||||
return { Authorization: `Token ${token}` };
|
||||
}
|
||||
return { Authorization: `Bearer ${token}` };
|
||||
function upstreamErrorResponse(res, errText) {
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a path segment to prevent path traversal / SSRF.
|
||||
*/
|
||||
function isValidPathSegment(segment: string): boolean {
|
||||
return !segment.includes("..") && !segment.includes("//");
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -37,21 +50,14 @@ async function handleDeepgramTranscription(providerConfig, file, modelId, token)
|
||||
const res = await fetch(url.toString(), {
|
||||
method: "POST",
|
||||
headers: {
|
||||
...buildAuthHeader(providerConfig, token),
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
"Content-Type": file.type || "audio/wav",
|
||||
},
|
||||
body: arrayBuffer,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
@@ -65,7 +71,7 @@ async function handleDeepgramTranscription(providerConfig, file, modelId, token)
|
||||
* Handle AssemblyAI transcription (async: upload file → submit → poll)
|
||||
*/
|
||||
async function handleAssemblyAITranscription(providerConfig, file, modelId, token) {
|
||||
const authHeaders = buildAuthHeader(providerConfig, token);
|
||||
const authHeaders = buildAuthHeaders(providerConfig, token);
|
||||
|
||||
// Step 1: Upload the audio file
|
||||
const arrayBuffer = await file.arrayBuffer();
|
||||
@@ -79,14 +85,7 @@ async function handleAssemblyAITranscription(providerConfig, file, modelId, toke
|
||||
});
|
||||
|
||||
if (!uploadRes.ok) {
|
||||
const errText = await uploadRes.text();
|
||||
return new Response(errText, {
|
||||
status: uploadRes.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(uploadRes, await uploadRes.text());
|
||||
}
|
||||
|
||||
const { upload_url } = await uploadRes.json();
|
||||
@@ -106,14 +105,7 @@ async function handleAssemblyAITranscription(providerConfig, file, modelId, toke
|
||||
});
|
||||
|
||||
if (!submitRes.ok) {
|
||||
const errText = await submitRes.text();
|
||||
return new Response(errText, {
|
||||
status: submitRes.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(submitRes, await submitRes.text());
|
||||
}
|
||||
|
||||
const { id: transcriptId } = await submitRes.json();
|
||||
@@ -146,6 +138,63 @@ async function handleAssemblyAITranscription(providerConfig, file, modelId, toke
|
||||
return errorResponse(504, "AssemblyAI transcription timed out after 120s");
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle Nvidia NIM transcription
|
||||
* Multipart POST, transform response to { text }
|
||||
*/
|
||||
async function handleNvidiaTranscription(providerConfig, file, modelId, token) {
|
||||
const upstreamForm = new FormData();
|
||||
upstreamForm.append("file", /** @type {Blob} */ file, /** @type {any} */ file.name || "audio.wav");
|
||||
upstreamForm.append("model", modelId);
|
||||
|
||||
const res = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: buildAuthHeaders(providerConfig, token),
|
||||
body: upstreamForm,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
// Normalize to { text } — Nvidia may return { text } directly or nested
|
||||
const text = data.text || data.transcript || "";
|
||||
|
||||
return Response.json({ text }, { headers: { "Access-Control-Allow-Origin": getCorsOrigin() } });
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle HuggingFace Inference transcription
|
||||
* POST raw binary audio to {baseUrl}/{model_id}, returns { text }
|
||||
*/
|
||||
async function handleHuggingFaceTranscription(providerConfig, file, modelId, token) {
|
||||
if (!isValidPathSegment(modelId)) {
|
||||
return errorResponse(400, "Invalid model ID");
|
||||
}
|
||||
const url = `${providerConfig.baseUrl}/${modelId}`;
|
||||
const arrayBuffer = await file.arrayBuffer();
|
||||
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
...buildAuthHeaders(providerConfig, token),
|
||||
"Content-Type": file.type || "audio/wav",
|
||||
},
|
||||
body: arrayBuffer,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
// HuggingFace returns { text } directly
|
||||
const text = data.text || "";
|
||||
|
||||
return Response.json({ text }, { headers: { "Access-Control-Allow-Origin": getCorsOrigin() } });
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle audio transcription request
|
||||
*
|
||||
@@ -172,12 +221,13 @@ export async function handleAudioTranscription({ formData, credentials }) {
|
||||
if (!providerConfig) {
|
||||
return errorResponse(
|
||||
400,
|
||||
`No transcription provider found for model "${model}". Available: openai, groq, deepgram, assemblyai`
|
||||
`No transcription provider found for model "${model}". Available: openai, groq, deepgram, assemblyai, nvidia, huggingface, qwen`
|
||||
);
|
||||
}
|
||||
|
||||
const token = credentials?.apiKey || credentials?.accessToken;
|
||||
if (!token) {
|
||||
// Skip credential check for local providers (authType: "none")
|
||||
const token = providerConfig.authType === "none" ? null : (credentials?.apiKey || credentials?.accessToken);
|
||||
if (providerConfig.authType !== "none" && !token) {
|
||||
return errorResponse(401, `No credentials for transcription provider: ${providerId}`);
|
||||
}
|
||||
|
||||
@@ -190,7 +240,15 @@ export async function handleAudioTranscription({ formData, credentials }) {
|
||||
return handleAssemblyAITranscription(providerConfig, file, modelId, token);
|
||||
}
|
||||
|
||||
// Default: OpenAI/Groq-compatible multipart proxy
|
||||
if (providerConfig.format === "nvidia-asr") {
|
||||
return handleNvidiaTranscription(providerConfig, file, modelId, token);
|
||||
}
|
||||
|
||||
if (providerConfig.format === "huggingface-asr") {
|
||||
return handleHuggingFaceTranscription(providerConfig, file, modelId, token);
|
||||
}
|
||||
|
||||
// Default: OpenAI/Groq/Qwen3-compatible multipart proxy
|
||||
const upstreamForm = new FormData();
|
||||
upstreamForm.append(
|
||||
"file",
|
||||
@@ -216,19 +274,12 @@ export async function handleAudioTranscription({ formData, credentials }) {
|
||||
try {
|
||||
const res = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: buildAuthHeader(providerConfig, token),
|
||||
headers: buildAuthHeaders(providerConfig, token),
|
||||
body: upstreamForm,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
return new Response(errText, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"Access-Control-Allow-Origin": getCorsOrigin(),
|
||||
},
|
||||
});
|
||||
return upstreamErrorResponse(res, await res.text());
|
||||
}
|
||||
|
||||
const data = await res.text();
|
||||
@@ -241,4 +292,4 @@ export async function handleAudioTranscription({ formData, credentials }) {
|
||||
} catch (err) {
|
||||
return errorResponse(500, `Transcription request failed: ${err.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -17,6 +17,12 @@
|
||||
|
||||
import { getImageProvider, parseImageModel } from "../config/imageRegistry.ts";
|
||||
import { saveCallLog } from "@/lib/usageDb";
|
||||
import {
|
||||
submitComfyWorkflow,
|
||||
pollComfyResult,
|
||||
fetchComfyOutput,
|
||||
extractComfyOutputFiles,
|
||||
} from "../utils/comfyuiClient.ts";
|
||||
|
||||
/**
|
||||
* Handle image generation request
|
||||
@@ -72,6 +78,14 @@ export async function handleImageGeneration({ body, credentials, log }) {
|
||||
});
|
||||
}
|
||||
|
||||
if (providerConfig.format === "sdwebui") {
|
||||
return handleSDWebUIImageGeneration({ model, provider, providerConfig, body, log });
|
||||
}
|
||||
|
||||
if (providerConfig.format === "comfyui") {
|
||||
return handleComfyUIImageGeneration({ model, provider, providerConfig, body, log });
|
||||
}
|
||||
|
||||
return handleOpenAIImageGeneration({ model, provider, providerConfig, body, credentials, log });
|
||||
}
|
||||
|
||||
@@ -560,3 +574,193 @@ async function handleNanoBananaImageGeneration({
|
||||
return { success: false, status: 502, error: `Image provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle SD WebUI image generation (local, no auth)
|
||||
* POST {baseUrl} with { prompt, negative_prompt, width, height, steps }
|
||||
* Response: { images: ["base64..."] }
|
||||
*/
|
||||
async function handleSDWebUIImageGeneration({ model, provider, providerConfig, body, log }) {
|
||||
const startTime = Date.now();
|
||||
const [width, height] = (body.size || "512x512").split("x").map(Number);
|
||||
|
||||
const upstreamBody = {
|
||||
prompt: body.prompt,
|
||||
negative_prompt: body.negative_prompt || "",
|
||||
width: width || 512,
|
||||
height: height || 512,
|
||||
steps: body.steps || 20,
|
||||
cfg_scale: body.cfg_scale || 7,
|
||||
sampler_name: body.sampler || "Euler a",
|
||||
batch_size: body.n || 1,
|
||||
override_settings: {
|
||||
sd_model_checkpoint: model,
|
||||
},
|
||||
};
|
||||
|
||||
if (log) {
|
||||
const promptPreview = String(body.prompt ?? "").slice(0, 60);
|
||||
log.info("IMAGE", `${provider}/${model} (sdwebui) | prompt: "${promptPreview}..."`);
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(upstreamBody),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
if (log) log.error("IMAGE", `${provider} error ${response.status}: ${errorText.slice(0, 200)}`);
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: response.status,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: errorText.slice(0, 500),
|
||||
}).catch(() => {});
|
||||
|
||||
return { success: false, status: response.status, error: errorText };
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
// SD WebUI returns { images: ["base64...", ...] }
|
||||
const images = (data.images || []).map((b64) => ({
|
||||
b64_json: b64,
|
||||
revised_prompt: body.prompt,
|
||||
}));
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: 200,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
responseBody: { images_count: images.length },
|
||||
}).catch(() => {});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { created: Math.floor(Date.now() / 1000), data: images },
|
||||
};
|
||||
} catch (err) {
|
||||
if (log) log.error("IMAGE", `${provider} sdwebui error: ${err.message}`);
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: 502,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: err.message,
|
||||
}).catch(() => {});
|
||||
return { success: false, status: 502, error: `Image provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle ComfyUI image generation (local, no auth)
|
||||
* Submits a txt2img workflow, polls for completion, fetches output
|
||||
*/
|
||||
async function handleComfyUIImageGeneration({ model, provider, providerConfig, body, log }) {
|
||||
const startTime = Date.now();
|
||||
const [width, height] = (body.size || "1024x1024").split("x").map(Number);
|
||||
|
||||
// Default txt2img workflow template for ComfyUI
|
||||
const workflow = {
|
||||
"3": {
|
||||
class_type: "KSampler",
|
||||
inputs: {
|
||||
seed: Math.floor(Math.random() * 2 ** 32),
|
||||
steps: body.steps || 20,
|
||||
cfg: body.cfg_scale || 7,
|
||||
sampler_name: "euler",
|
||||
scheduler: "normal",
|
||||
denoise: 1,
|
||||
model: ["4", 0],
|
||||
positive: ["6", 0],
|
||||
negative: ["7", 0],
|
||||
latent_image: ["5", 0],
|
||||
},
|
||||
},
|
||||
"4": {
|
||||
class_type: "CheckpointLoaderSimple",
|
||||
inputs: { ckpt_name: model },
|
||||
},
|
||||
"5": {
|
||||
class_type: "EmptyLatentImage",
|
||||
inputs: { width: width || 1024, height: height || 1024, batch_size: body.n || 1 },
|
||||
},
|
||||
"6": {
|
||||
class_type: "CLIPTextEncode",
|
||||
inputs: { text: body.prompt, clip: ["4", 1] },
|
||||
},
|
||||
"7": {
|
||||
class_type: "CLIPTextEncode",
|
||||
inputs: { text: body.negative_prompt || "", clip: ["4", 1] },
|
||||
},
|
||||
"8": {
|
||||
class_type: "VAEDecode",
|
||||
inputs: { samples: ["3", 0], vae: ["4", 2] },
|
||||
},
|
||||
"9": {
|
||||
class_type: "SaveImage",
|
||||
inputs: { filename_prefix: "omniroute", images: ["8", 0] },
|
||||
},
|
||||
};
|
||||
|
||||
if (log) {
|
||||
const promptPreview = String(body.prompt ?? "").slice(0, 60);
|
||||
log.info("IMAGE", `${provider}/${model} (comfyui) | prompt: "${promptPreview}..."`);
|
||||
}
|
||||
|
||||
try {
|
||||
const promptId = await submitComfyWorkflow(providerConfig.baseUrl, workflow);
|
||||
const historyEntry = await pollComfyResult(providerConfig.baseUrl, promptId);
|
||||
const outputFiles = extractComfyOutputFiles(historyEntry);
|
||||
|
||||
const images = [];
|
||||
for (const file of outputFiles) {
|
||||
const buffer = await fetchComfyOutput(
|
||||
providerConfig.baseUrl,
|
||||
file.filename,
|
||||
file.subfolder,
|
||||
file.type
|
||||
);
|
||||
const base64 = Buffer.from(buffer).toString("base64");
|
||||
images.push({ b64_json: base64, revised_prompt: body.prompt });
|
||||
}
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: 200,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
responseBody: { images_count: images.length },
|
||||
}).catch(() => {});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { created: Math.floor(Date.now() / 1000), data: images },
|
||||
};
|
||||
} catch (err) {
|
||||
if (log) log.error("IMAGE", `${provider} comfyui error: ${err.message}`);
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: 502,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: err.message,
|
||||
}).catch(() => {});
|
||||
return { success: false, status: 502, error: `Image provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
/**
|
||||
* Music Generation Handler
|
||||
*
|
||||
* Handles POST /v1/music/generations requests.
|
||||
* Proxies to upstream music generation providers.
|
||||
*
|
||||
* Supported provider formats:
|
||||
* - ComfyUI: submit audio workflow → poll → fetch output
|
||||
*
|
||||
* Response format (OpenAI-like):
|
||||
* {
|
||||
* "created": 1234567890,
|
||||
* "data": [{ "b64_json": "...", "format": "wav" }]
|
||||
* }
|
||||
*/
|
||||
|
||||
import { getMusicProvider, parseMusicModel } from "../config/musicRegistry.ts";
|
||||
import {
|
||||
submitComfyWorkflow,
|
||||
pollComfyResult,
|
||||
fetchComfyOutput,
|
||||
extractComfyOutputFiles,
|
||||
} from "../utils/comfyuiClient.ts";
|
||||
import { saveCallLog } from "@/lib/usageDb";
|
||||
|
||||
/**
|
||||
* Handle music generation request
|
||||
*/
|
||||
export async function handleMusicGeneration({ body, credentials, log }) {
|
||||
const { provider, model } = parseMusicModel(body.model);
|
||||
|
||||
if (!provider) {
|
||||
return {
|
||||
success: false,
|
||||
status: 400,
|
||||
error: `Invalid music model: ${body.model}. Use format: provider/model`,
|
||||
};
|
||||
}
|
||||
|
||||
const providerConfig = getMusicProvider(provider);
|
||||
if (!providerConfig) {
|
||||
return {
|
||||
success: false,
|
||||
status: 400,
|
||||
error: `Unknown music provider: ${provider}`,
|
||||
};
|
||||
}
|
||||
|
||||
if (providerConfig.format === "comfyui") {
|
||||
return handleComfyUIMusicGeneration({ model, provider, providerConfig, body, log });
|
||||
}
|
||||
|
||||
return { success: false, status: 400, error: `Unsupported music format: ${providerConfig.format}` };
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle ComfyUI music generation
|
||||
* Submits an audio generation workflow (Stable Audio / MusicGen), polls, fetches output
|
||||
*/
|
||||
async function handleComfyUIMusicGeneration({ model, provider, providerConfig, body, log }) {
|
||||
const startTime = Date.now();
|
||||
const duration = body.duration || 10; // seconds
|
||||
|
||||
// Audio generation workflow template for ComfyUI
|
||||
const workflow = {
|
||||
"1": {
|
||||
class_type: "CheckpointLoaderSimple",
|
||||
inputs: { ckpt_name: model },
|
||||
},
|
||||
"2": {
|
||||
class_type: "CLIPTextEncode",
|
||||
inputs: { text: body.prompt, clip: ["1", 1] },
|
||||
},
|
||||
"3": {
|
||||
class_type: "CLIPTextEncode",
|
||||
inputs: { text: body.negative_prompt || "", clip: ["1", 1] },
|
||||
},
|
||||
"4": {
|
||||
class_type: "EmptyLatentAudio",
|
||||
inputs: { seconds: duration },
|
||||
},
|
||||
"5": {
|
||||
class_type: "KSampler",
|
||||
inputs: {
|
||||
seed: Math.floor(Math.random() * 2 ** 32),
|
||||
steps: body.steps || 100,
|
||||
cfg: body.cfg_scale || 7,
|
||||
sampler_name: "euler",
|
||||
scheduler: "normal",
|
||||
denoise: 1,
|
||||
model: ["1", 0],
|
||||
positive: ["2", 0],
|
||||
negative: ["3", 0],
|
||||
latent_image: ["4", 0],
|
||||
},
|
||||
},
|
||||
"6": {
|
||||
class_type: "VAEDecodeAudio",
|
||||
inputs: { samples: ["5", 0], vae: ["1", 2] },
|
||||
},
|
||||
"7": {
|
||||
class_type: "SaveAudio",
|
||||
inputs: {
|
||||
filename_prefix: "omniroute_music",
|
||||
audio: ["6", 0],
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
if (log) {
|
||||
const promptPreview = String(body.prompt ?? "").slice(0, 60);
|
||||
log.info("MUSIC", `${provider}/${model} (comfyui) | prompt: "${promptPreview}..." | duration: ${duration}s`);
|
||||
}
|
||||
|
||||
try {
|
||||
const promptId = await submitComfyWorkflow(providerConfig.baseUrl, workflow);
|
||||
const historyEntry = await pollComfyResult(providerConfig.baseUrl, promptId, 300_000);
|
||||
const outputFiles = extractComfyOutputFiles(historyEntry);
|
||||
|
||||
const audioFiles = [];
|
||||
for (const file of outputFiles) {
|
||||
const buffer = await fetchComfyOutput(
|
||||
providerConfig.baseUrl,
|
||||
file.filename,
|
||||
file.subfolder,
|
||||
file.type
|
||||
);
|
||||
const base64 = Buffer.from(buffer).toString("base64");
|
||||
audioFiles.push({ b64_json: base64, format: "wav" });
|
||||
}
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/music/generations",
|
||||
status: 200,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
responseBody: { audio_count: audioFiles.length },
|
||||
}).catch(() => {});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { created: Math.floor(Date.now() / 1000), data: audioFiles },
|
||||
};
|
||||
} catch (err) {
|
||||
if (log) log.error("MUSIC", `${provider} comfyui error: ${err.message}`);
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/music/generations",
|
||||
status: 502,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: err.message,
|
||||
}).catch(() => {});
|
||||
return { success: false, status: 502, error: `Music provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,256 @@
|
||||
/**
|
||||
* Video Generation Handler
|
||||
*
|
||||
* Handles POST /v1/videos/generations requests.
|
||||
* Proxies to upstream video generation providers.
|
||||
*
|
||||
* Supported provider formats:
|
||||
* - ComfyUI: submit AnimateDiff/SVD workflow → poll → fetch video
|
||||
* - SD WebUI: POST to AnimateDiff extension endpoint
|
||||
*
|
||||
* Response format (OpenAI-like):
|
||||
* {
|
||||
* "created": 1234567890,
|
||||
* "data": [{ "b64_json": "...", "format": "mp4" }]
|
||||
* }
|
||||
*/
|
||||
|
||||
import { getVideoProvider, parseVideoModel } from "../config/videoRegistry.ts";
|
||||
import {
|
||||
submitComfyWorkflow,
|
||||
pollComfyResult,
|
||||
fetchComfyOutput,
|
||||
extractComfyOutputFiles,
|
||||
} from "../utils/comfyuiClient.ts";
|
||||
import { saveCallLog } from "@/lib/usageDb";
|
||||
|
||||
/**
|
||||
* Handle video generation request
|
||||
*/
|
||||
export async function handleVideoGeneration({ body, credentials, log }) {
|
||||
const { provider, model } = parseVideoModel(body.model);
|
||||
|
||||
if (!provider) {
|
||||
return {
|
||||
success: false,
|
||||
status: 400,
|
||||
error: `Invalid video model: ${body.model}. Use format: provider/model`,
|
||||
};
|
||||
}
|
||||
|
||||
const providerConfig = getVideoProvider(provider);
|
||||
if (!providerConfig) {
|
||||
return {
|
||||
success: false,
|
||||
status: 400,
|
||||
error: `Unknown video provider: ${provider}`,
|
||||
};
|
||||
}
|
||||
|
||||
if (providerConfig.format === "comfyui") {
|
||||
return handleComfyUIVideoGeneration({ model, provider, providerConfig, body, log });
|
||||
}
|
||||
|
||||
if (providerConfig.format === "sdwebui-video") {
|
||||
return handleSDWebUIVideoGeneration({ model, provider, providerConfig, body, log });
|
||||
}
|
||||
|
||||
return { success: false, status: 400, error: `Unsupported video format: ${providerConfig.format}` };
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle ComfyUI video generation
|
||||
* Submits an AnimateDiff or SVD workflow, polls for completion, fetches output video
|
||||
*/
|
||||
async function handleComfyUIVideoGeneration({ model, provider, providerConfig, body, log }) {
|
||||
const startTime = Date.now();
|
||||
const [width, height] = (body.size || "512x512").split("x").map(Number);
|
||||
const frames = body.frames || 16;
|
||||
|
||||
// AnimateDiff workflow template
|
||||
const workflow = {
|
||||
"1": {
|
||||
class_type: "CheckpointLoaderSimple",
|
||||
inputs: { ckpt_name: model },
|
||||
},
|
||||
"2": {
|
||||
class_type: "CLIPTextEncode",
|
||||
inputs: { text: body.prompt, clip: ["1", 1] },
|
||||
},
|
||||
"3": {
|
||||
class_type: "CLIPTextEncode",
|
||||
inputs: { text: body.negative_prompt || "", clip: ["1", 1] },
|
||||
},
|
||||
"4": {
|
||||
class_type: "EmptyLatentImage",
|
||||
inputs: { width: width || 512, height: height || 512, batch_size: frames },
|
||||
},
|
||||
"5": {
|
||||
class_type: "KSampler",
|
||||
inputs: {
|
||||
seed: Math.floor(Math.random() * 2 ** 32),
|
||||
steps: body.steps || 20,
|
||||
cfg: body.cfg_scale || 7,
|
||||
sampler_name: "euler",
|
||||
scheduler: "normal",
|
||||
denoise: 1,
|
||||
model: ["1", 0],
|
||||
positive: ["2", 0],
|
||||
negative: ["3", 0],
|
||||
latent_image: ["4", 0],
|
||||
},
|
||||
},
|
||||
"6": {
|
||||
class_type: "VAEDecode",
|
||||
inputs: { samples: ["5", 0], vae: ["1", 2] },
|
||||
},
|
||||
"7": {
|
||||
class_type: "SaveAnimatedWEBP",
|
||||
inputs: {
|
||||
filename_prefix: "omniroute_video",
|
||||
fps: body.fps || 8,
|
||||
lossless: false,
|
||||
quality: 80,
|
||||
method: "default",
|
||||
images: ["6", 0],
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
if (log) {
|
||||
const promptPreview = String(body.prompt ?? "").slice(0, 60);
|
||||
log.info("VIDEO", `${provider}/${model} (comfyui) | prompt: "${promptPreview}..." | frames: ${frames}`);
|
||||
}
|
||||
|
||||
try {
|
||||
const promptId = await submitComfyWorkflow(providerConfig.baseUrl, workflow);
|
||||
const historyEntry = await pollComfyResult(providerConfig.baseUrl, promptId, 300_000);
|
||||
const outputFiles = extractComfyOutputFiles(historyEntry);
|
||||
|
||||
const videos = [];
|
||||
for (const file of outputFiles) {
|
||||
const buffer = await fetchComfyOutput(
|
||||
providerConfig.baseUrl,
|
||||
file.filename,
|
||||
file.subfolder,
|
||||
file.type
|
||||
);
|
||||
const base64 = Buffer.from(buffer).toString("base64");
|
||||
videos.push({ b64_json: base64, format: "webp" });
|
||||
}
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/videos/generations",
|
||||
status: 200,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
responseBody: { videos_count: videos.length },
|
||||
}).catch(() => {});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { created: Math.floor(Date.now() / 1000), data: videos },
|
||||
};
|
||||
} catch (err) {
|
||||
if (log) log.error("VIDEO", `${provider} comfyui error: ${err.message}`);
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/videos/generations",
|
||||
status: 502,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: err.message,
|
||||
}).catch(() => {});
|
||||
return { success: false, status: 502, error: `Video provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle SD WebUI video generation via AnimateDiff extension
|
||||
* POST to the AnimateDiff API endpoint
|
||||
*/
|
||||
async function handleSDWebUIVideoGeneration({ model, provider, providerConfig, body, log }) {
|
||||
const startTime = Date.now();
|
||||
const [width, height] = (body.size || "512x512").split("x").map(Number);
|
||||
const url = `${providerConfig.baseUrl}/animatediff/v1/generate`;
|
||||
|
||||
const upstreamBody = {
|
||||
prompt: body.prompt,
|
||||
negative_prompt: body.negative_prompt || "",
|
||||
width: width || 512,
|
||||
height: height || 512,
|
||||
steps: body.steps || 20,
|
||||
cfg_scale: body.cfg_scale || 7,
|
||||
frames: body.frames || 16,
|
||||
fps: body.fps || 8,
|
||||
};
|
||||
|
||||
if (log) {
|
||||
const promptPreview = String(body.prompt ?? "").slice(0, 60);
|
||||
log.info("VIDEO", `${provider}/${model} (sdwebui) | prompt: "${promptPreview}..."`);
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(upstreamBody),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
if (log) log.error("VIDEO", `${provider} error ${response.status}: ${errorText.slice(0, 200)}`);
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/videos/generations",
|
||||
status: response.status,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: errorText.slice(0, 500),
|
||||
}).catch(() => {});
|
||||
return { success: false, status: response.status, error: errorText };
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
// SD WebUI AnimateDiff returns { video: "base64..." } or { images: [...] }
|
||||
const videos = [];
|
||||
if (data.video) {
|
||||
videos.push({ b64_json: data.video, format: "mp4" });
|
||||
} else if (data.images) {
|
||||
for (const img of data.images) {
|
||||
videos.push({ b64_json: typeof img === "string" ? img : img.image, format: "mp4" });
|
||||
}
|
||||
}
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/videos/generations",
|
||||
status: 200,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
responseBody: { videos_count: videos.length },
|
||||
}).catch(() => {});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { created: Math.floor(Date.now() / 1000), data: videos },
|
||||
};
|
||||
} catch (err) {
|
||||
if (log) log.error("VIDEO", `${provider} sdwebui error: ${err.message}`);
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/videos/generations",
|
||||
status: 502,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: err.message,
|
||||
}).catch(() => {});
|
||||
return { success: false, status: 502, error: `Video provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
@@ -139,3 +139,28 @@ export {
|
||||
parseModerationModel,
|
||||
getAllModerationModels,
|
||||
} from "./config/moderationRegistry.ts";
|
||||
|
||||
// Video Generation
|
||||
export { handleVideoGeneration } from "./handlers/videoGeneration.ts";
|
||||
export {
|
||||
VIDEO_PROVIDERS,
|
||||
getVideoProvider,
|
||||
parseVideoModel,
|
||||
getAllVideoModels,
|
||||
} from "./config/videoRegistry.ts";
|
||||
|
||||
// Music Generation
|
||||
export { handleMusicGeneration } from "./handlers/musicGeneration.ts";
|
||||
export {
|
||||
MUSIC_PROVIDERS,
|
||||
getMusicProvider,
|
||||
parseMusicModel,
|
||||
getAllMusicModels,
|
||||
} from "./config/musicRegistry.ts";
|
||||
|
||||
// Registry Utilities
|
||||
export {
|
||||
parseModelFromRegistry,
|
||||
getAllModelsFromRegistry,
|
||||
buildAuthHeaders,
|
||||
} from "./config/registryUtils.ts";
|
||||
|
||||
@@ -24,14 +24,25 @@ export function getProviderProfile(provider) {
|
||||
// In-memory map: "provider:connectionId:model" → { reason, until, lockedAt }
|
||||
const modelLockouts = new Map();
|
||||
|
||||
// Auto-cleanup expired lockouts every 15 seconds
|
||||
const _cleanupTimer = setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [key, entry] of modelLockouts) {
|
||||
if (now > entry.until) modelLockouts.delete(key);
|
||||
// Auto-cleanup expired lockouts every 15 seconds (lazy init for Cloudflare Workers compatibility)
|
||||
let _cleanupTimer: ReturnType<typeof setInterval> | null = null;
|
||||
|
||||
function ensureCleanupTimer() {
|
||||
if (_cleanupTimer) return;
|
||||
try {
|
||||
_cleanupTimer = setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [key, entry] of modelLockouts) {
|
||||
if (now > entry.until) modelLockouts.delete(key);
|
||||
}
|
||||
}, 15_000);
|
||||
if (typeof _cleanupTimer === "object" && "unref" in _cleanupTimer) {
|
||||
(_cleanupTimer as any).unref(); // Don't prevent process exit (Node.js only)
|
||||
}
|
||||
} catch {
|
||||
// Cloudflare Workers may not support setInterval outside handlers — skip cleanup timer
|
||||
}
|
||||
}, 15_000);
|
||||
_cleanupTimer.unref(); // Don't prevent process exit
|
||||
}
|
||||
|
||||
/**
|
||||
* Lock a specific model on a specific account
|
||||
@@ -43,6 +54,7 @@ _cleanupTimer.unref(); // Don't prevent process exit
|
||||
*/
|
||||
export function lockModel(provider, connectionId, model, reason, cooldownMs) {
|
||||
if (!model) return; // No model → skip model-level locking
|
||||
ensureCleanupTimer();
|
||||
const key = `${provider}:${connectionId}:${model}`;
|
||||
modelLockouts.set(key, {
|
||||
reason,
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
/**
|
||||
* Background Task Detector — Feature 3
|
||||
*
|
||||
* Detects when CLI tools send "background" requests (title generation,
|
||||
* summarization, short descriptions) and provides model degradation
|
||||
* recommendations to save premium model quota.
|
||||
*
|
||||
* Detection heuristics:
|
||||
* - System prompt patterns indicating background/utility tasks
|
||||
* - Very short conversations with summary-like system prompts
|
||||
* - X-Request-Priority header
|
||||
*/
|
||||
|
||||
// ── Configuration ───────────────────────────────────────────────────────────
|
||||
|
||||
interface DegradationConfig {
|
||||
enabled: boolean;
|
||||
degradationMap: Record<string, string>; // original → cheaper model
|
||||
detectionPatterns: string[]; // regex patterns for system prompt matching
|
||||
stats: {
|
||||
detected: number;
|
||||
tokensSaved: number;
|
||||
};
|
||||
}
|
||||
|
||||
const DEFAULT_DETECTION_PATTERNS = [
|
||||
"generate a title",
|
||||
"generate title",
|
||||
"create a title",
|
||||
"create a short",
|
||||
"summarize this",
|
||||
"summarize the",
|
||||
"write a brief",
|
||||
"write a summary",
|
||||
"one-line summary",
|
||||
"one line summary",
|
||||
"short description",
|
||||
"brief description",
|
||||
"conversation title",
|
||||
"chat title",
|
||||
"name this conversation",
|
||||
"name this chat",
|
||||
"title for this",
|
||||
"suggest a title",
|
||||
"label this",
|
||||
];
|
||||
|
||||
const DEFAULT_DEGRADATION_MAP: Record<string, string> = {
|
||||
// Premium → Cheap alternatives
|
||||
"claude-opus-4-6": "gemini-2.5-flash",
|
||||
"claude-opus-4-6-thinking": "gemini-2.5-flash",
|
||||
"claude-opus-4-5-20251101": "gemini-2.5-flash",
|
||||
"claude-sonnet-4-5-20250929": "gemini-2.5-flash",
|
||||
"claude-sonnet-4-20250514": "gemini-2.5-flash",
|
||||
"claude-sonnet-4": "gemini-2.5-flash",
|
||||
"gemini-3.1-pro": "gemini-3.1-flash",
|
||||
"gemini-3.1-pro-high": "gemini-3.1-flash",
|
||||
"gemini-3-pro-preview": "gemini-3-flash-preview",
|
||||
"gemini-2.5-pro": "gemini-2.5-flash",
|
||||
"gpt-4o": "gpt-4o-mini",
|
||||
"gpt-5": "gpt-5-mini",
|
||||
"gpt-5.1": "gpt-5-mini",
|
||||
"gpt-5.1-codex": "gpt-5.1-codex-mini",
|
||||
};
|
||||
|
||||
// ── State ───────────────────────────────────────────────────────────────────
|
||||
|
||||
let _config: DegradationConfig = {
|
||||
enabled: false, // Disabled by default — user must opt in
|
||||
degradationMap: { ...DEFAULT_DEGRADATION_MAP },
|
||||
detectionPatterns: [...DEFAULT_DETECTION_PATTERNS],
|
||||
stats: { detected: 0, tokensSaved: 0 },
|
||||
};
|
||||
|
||||
// ── Config Management ───────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Set the background degradation config (called from settings API or startup).
|
||||
*/
|
||||
export function setBackgroundDegradationConfig(config: Partial<DegradationConfig>): void {
|
||||
_config = {
|
||||
..._config,
|
||||
...config,
|
||||
stats: _config.stats, // preserve stats across config changes
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current background degradation config.
|
||||
*/
|
||||
export function getBackgroundDegradationConfig(): DegradationConfig {
|
||||
return {
|
||||
..._config,
|
||||
degradationMap: { ..._config.degradationMap },
|
||||
detectionPatterns: [..._config.detectionPatterns],
|
||||
stats: { ..._config.stats },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset stats counters.
|
||||
*/
|
||||
export function resetStats(): void {
|
||||
_config.stats = { detected: 0, tokensSaved: 0 };
|
||||
}
|
||||
|
||||
// ── Detection ───────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Check if a request is a background/utility task.
|
||||
*
|
||||
* @param {object} body - Request body
|
||||
* @param {object} [headers] - Request headers (optional)
|
||||
* @returns {boolean} True if the request looks like a background task
|
||||
*/
|
||||
export function isBackgroundTask(
|
||||
body: any,
|
||||
headers: Record<string, string> | null = null
|
||||
): boolean {
|
||||
if (!body || typeof body !== "object") return false;
|
||||
|
||||
// 1. Check explicit header
|
||||
if (headers) {
|
||||
const priority =
|
||||
headers["x-request-priority"] || headers["X-Request-Priority"] || headers["x-initiator"];
|
||||
if (priority === "background" || priority === "Background") return true;
|
||||
}
|
||||
|
||||
// 2. Check system prompt for background task patterns
|
||||
const messages = body.messages || body.input || [];
|
||||
if (!Array.isArray(messages) || messages.length === 0) return false;
|
||||
|
||||
// Find system message
|
||||
const systemMsg = messages.find((m: any) => m.role === "system" || m.role === "developer");
|
||||
if (!systemMsg) return false;
|
||||
|
||||
const systemContent =
|
||||
typeof systemMsg.content === "string" ? systemMsg.content.toLowerCase() : "";
|
||||
|
||||
if (!systemContent) return false;
|
||||
|
||||
// Check against detection patterns
|
||||
const matched = _config.detectionPatterns.some((pattern) =>
|
||||
systemContent.includes(pattern.toLowerCase())
|
||||
);
|
||||
|
||||
if (!matched) return false;
|
||||
|
||||
// 3. Additional heuristic: background tasks typically have very few messages
|
||||
// (system + 1-2 user messages)
|
||||
const userMessages = messages.filter((m: any) => m.role === "user");
|
||||
if (userMessages.length > 3) return false; // Too many turns for a background task
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the degraded (cheaper) model for a given model.
|
||||
*
|
||||
* @param {string} originalModel - The original model ID
|
||||
* @returns {string} The cheaper model or original if no mapping exists
|
||||
*/
|
||||
export function getDegradedModel(originalModel: string): string {
|
||||
if (!originalModel) return originalModel;
|
||||
|
||||
const degraded = _config.degradationMap[originalModel];
|
||||
if (degraded) {
|
||||
_config.stats.detected++;
|
||||
return degraded;
|
||||
}
|
||||
|
||||
return originalModel;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get default degradation map (for UI reset).
|
||||
*/
|
||||
export function getDefaultDegradationMap(): Record<string, string> {
|
||||
return { ...DEFAULT_DEGRADATION_MAP };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get default detection patterns (for UI reset).
|
||||
*/
|
||||
export function getDefaultDetectionPatterns(): string[] {
|
||||
return [...DEFAULT_DETECTION_PATTERNS];
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
/**
|
||||
* Model Deprecation Auto-Forward — Feature 2
|
||||
*
|
||||
* Maps deprecated model IDs to their replacements so user configs
|
||||
* don't break when providers rename or retire models.
|
||||
*
|
||||
* Supports both built-in aliases (static) and custom aliases (persisted via Settings API).
|
||||
*/
|
||||
|
||||
// ── Built-in Deprecation Aliases ────────────────────────────────────────────
|
||||
// These are known renames/retirements across providers.
|
||||
// Format: deprecated ID → current ID
|
||||
const BUILT_IN_ALIASES: Record<string, string> = {
|
||||
// Gemini legacy → current
|
||||
"gemini-pro": "gemini-2.5-pro",
|
||||
"gemini-pro-vision": "gemini-2.5-pro",
|
||||
"gemini-1.5-pro": "gemini-2.5-pro",
|
||||
"gemini-1.5-flash": "gemini-2.5-flash",
|
||||
"gemini-1.0-pro": "gemini-2.5-pro",
|
||||
"gemini-2.0-flash": "gemini-2.5-flash",
|
||||
|
||||
// Claude legacy → current
|
||||
"claude-3-opus-20240229": "claude-opus-4-20250514",
|
||||
"claude-3-sonnet-20240229": "claude-sonnet-4-20250514",
|
||||
"claude-3-haiku-20240307": "claude-3-5-sonnet-20241022",
|
||||
"claude-3-5-sonnet-latest": "claude-sonnet-4-20250514",
|
||||
"claude-3-5-haiku-latest": "claude-3-5-sonnet-20241022",
|
||||
|
||||
// OpenAI legacy → current
|
||||
"gpt-4-turbo-preview": "gpt-4-turbo",
|
||||
"gpt-4-0125-preview": "gpt-4-turbo",
|
||||
"gpt-4-1106-preview": "gpt-4-turbo",
|
||||
"gpt-3.5-turbo-0125": "gpt-3.5-turbo",
|
||||
};
|
||||
|
||||
// ── Custom Aliases (persisted via Settings API) ─────────────────────────────
|
||||
let _customAliases: Record<string, string> = {};
|
||||
|
||||
/**
|
||||
* Set custom aliases (called from settings API or startup).
|
||||
*/
|
||||
export function setCustomAliases(aliases: Record<string, string>): void {
|
||||
_customAliases = { ...aliases };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current custom aliases.
|
||||
*/
|
||||
export function getCustomAliases(): Record<string, string> {
|
||||
return { ..._customAliases };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the full alias map (built-in + custom).
|
||||
* Custom aliases take precedence over built-in.
|
||||
*/
|
||||
export function getAllAliases(): Record<string, string> {
|
||||
return { ...BUILT_IN_ALIASES, ..._customAliases };
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a model alias to its current ID.
|
||||
* Custom aliases override built-in ones.
|
||||
*
|
||||
* @param {string} modelId - The model ID to resolve
|
||||
* @returns {string} The resolved model ID, or the original if not deprecated
|
||||
*/
|
||||
export function resolveModelAlias(modelId: string): string {
|
||||
if (!modelId) return modelId;
|
||||
|
||||
// Check custom aliases first (higher priority)
|
||||
if (_customAliases[modelId]) return _customAliases[modelId];
|
||||
|
||||
// Then check built-in
|
||||
if (BUILT_IN_ALIASES[modelId]) return BUILT_IN_ALIASES[modelId];
|
||||
|
||||
return modelId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a deprecation notice if the model is deprecated.
|
||||
*
|
||||
* @param {string} modelId - The model ID to check
|
||||
* @returns {string | null} Deprecation message or null if not deprecated
|
||||
*/
|
||||
export function getDeprecationNotice(modelId: string): string | null {
|
||||
if (!modelId) return null;
|
||||
|
||||
const resolved = resolveModelAlias(modelId);
|
||||
if (resolved === modelId) return null;
|
||||
|
||||
return `Model "${modelId}" is deprecated. Forwarding to "${resolved}".`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a model is deprecated.
|
||||
*/
|
||||
export function isDeprecated(modelId: string): boolean {
|
||||
return getDeprecationNotice(modelId) !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Add a custom alias.
|
||||
*/
|
||||
export function addCustomAlias(from: string, to: string): void {
|
||||
_customAliases[from] = to;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a custom alias.
|
||||
*/
|
||||
export function removeCustomAlias(from: string): boolean {
|
||||
if (_customAliases[from]) {
|
||||
delete _customAliases[from];
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the built-in aliases (read-only reference).
|
||||
*/
|
||||
export function getBuiltInAliases(): Record<string, string> {
|
||||
return { ...BUILT_IN_ALIASES };
|
||||
}
|
||||
@@ -19,6 +19,11 @@ const limiters = new Map();
|
||||
// Store connections that have rate limit protection enabled
|
||||
const enabledConnections = new Set();
|
||||
|
||||
// Store learned limits for persistence (debounced)
|
||||
const learnedLimits: Record<string, any> = {};
|
||||
let persistTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
const PERSIST_DEBOUNCE_MS = 60_000; // Debounce persistence to every 60s max
|
||||
|
||||
// Track initialization
|
||||
let initialized = false;
|
||||
|
||||
@@ -82,6 +87,9 @@ export async function initializeRateLimits() {
|
||||
`🛡️ [RATE-LIMIT] Loaded ${explicitCount} explicit + ${autoCount} auto-enabled (API key) protection(s)`
|
||||
);
|
||||
}
|
||||
|
||||
// Load persisted learned limits
|
||||
await loadPersistedLimits();
|
||||
} catch (err) {
|
||||
console.error("[RATE-LIMIT] Failed to load settings:", err.message);
|
||||
}
|
||||
@@ -314,6 +322,9 @@ export function updateFromHeaders(provider, connectionId, headers, status, model
|
||||
}
|
||||
|
||||
limiter.updateSettings(updates);
|
||||
|
||||
// Persist learned limits (debounced)
|
||||
recordLearnedLimit(provider, connectionId, { limit, remaining, minTime: updates.minTime });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -360,6 +371,82 @@ export function getAllRateLimitStatus() {
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all learned limits (for dashboard display).
|
||||
*/
|
||||
export function getLearnedLimits() {
|
||||
return { ...learnedLimits };
|
||||
}
|
||||
|
||||
// ─── Persistence ────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Record a learned limit for debounced persistence.
|
||||
*/
|
||||
function recordLearnedLimit(provider: string, connectionId: string, limits: any) {
|
||||
const key = `${provider}:${connectionId}`;
|
||||
learnedLimits[key] = {
|
||||
...limits,
|
||||
provider,
|
||||
connectionId,
|
||||
lastUpdated: Date.now(),
|
||||
};
|
||||
|
||||
// Debounce: save at most once per PERSIST_DEBOUNCE_MS
|
||||
if (!persistTimer) {
|
||||
persistTimer = setTimeout(async () => {
|
||||
persistTimer = null;
|
||||
try {
|
||||
const { updateSettings } = await import("@/lib/db/settings");
|
||||
await updateSettings({ learnedRateLimits: JSON.stringify(learnedLimits) });
|
||||
console.log(
|
||||
`💾 [RATE-LIMIT] Persisted learned limits for ${Object.keys(learnedLimits).length} provider(s)`
|
||||
);
|
||||
} catch (err) {
|
||||
console.error("[RATE-LIMIT] Failed to persist learned limits:", err.message);
|
||||
}
|
||||
}, PERSIST_DEBOUNCE_MS);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load persisted learned limits on startup.
|
||||
*/
|
||||
async function loadPersistedLimits() {
|
||||
try {
|
||||
const { getSettings } = await import("@/lib/db/settings");
|
||||
const settings = await getSettings();
|
||||
const raw = settings?.learnedRateLimits;
|
||||
if (!raw) return;
|
||||
|
||||
const parsed = JSON.parse(raw);
|
||||
let count = 0;
|
||||
|
||||
for (const [key, data] of Object.entries<any>(parsed)) {
|
||||
// Skip stale entries (older than 24h)
|
||||
if (data.lastUpdated && Date.now() - data.lastUpdated > 24 * 60 * 60 * 1000) continue;
|
||||
|
||||
learnedLimits[key] = data;
|
||||
|
||||
// Apply to limiter if it exists and has rate limit enabled
|
||||
if (data.connectionId && enabledConnections.has(data.connectionId)) {
|
||||
const limiter = limiters.get(key);
|
||||
if (limiter && data.limit) {
|
||||
const minTime = data.minTime || Math.max(0, Math.floor(60000 / data.limit) - 10);
|
||||
limiter.updateSettings({ minTime });
|
||||
count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (count > 0) {
|
||||
console.log(`📥 [RATE-LIMIT] Restored ${count} learned rate limit(s) from persistence`);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("[RATE-LIMIT] Failed to load persisted limits:", err.message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Update rate limiter based on API response body (JSON error responses).
|
||||
* Providers embed retry info in JSON payloads in different formats.
|
||||
|
||||
@@ -21,6 +21,15 @@ export const EFFORT_BUDGETS = {
|
||||
high: 131072,
|
||||
};
|
||||
|
||||
// thinkingLevel string → budget token mapping
|
||||
// Used when clients send string-based thinking levels (e.g., VS Code Copilot)
|
||||
export const THINKING_LEVEL_MAP = {
|
||||
none: 0,
|
||||
low: 1024,
|
||||
medium: 10240,
|
||||
high: 131072,
|
||||
};
|
||||
|
||||
// Default config (passthrough = backward compatible)
|
||||
export const DEFAULT_THINKING_CONFIG = {
|
||||
mode: ThinkingMode.PASSTHROUGH,
|
||||
@@ -45,10 +54,81 @@ export function getThinkingBudgetConfig() {
|
||||
return { ..._config };
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize thinkingLevel string fields into numeric budget.
|
||||
* Handles: body.thinkingLevel, body.thinking_level,
|
||||
* and Gemini's generationConfig.thinkingConfig.thinkingLevel
|
||||
*
|
||||
* @param {object} body - Request body
|
||||
* @returns {object} Body with string thinkingLevel converted to numeric budget
|
||||
*/
|
||||
export function normalizeThinkingLevel(body) {
|
||||
if (!body || typeof body !== "object") return body;
|
||||
const result = { ...body };
|
||||
|
||||
// Handle top-level thinkingLevel or thinking_level string fields
|
||||
const levelStr = result.thinkingLevel || result.thinking_level;
|
||||
if (typeof levelStr === "string" && THINKING_LEVEL_MAP[levelStr] !== undefined) {
|
||||
const budget = THINKING_LEVEL_MAP[levelStr];
|
||||
// Convert to Claude thinking format as canonical representation
|
||||
result.thinking = {
|
||||
type: budget > 0 ? "enabled" : "disabled",
|
||||
budget_tokens: budget,
|
||||
};
|
||||
delete result.thinkingLevel;
|
||||
delete result.thinking_level;
|
||||
}
|
||||
|
||||
// Handle Gemini's generationConfig.thinkingConfig.thinkingLevel
|
||||
const geminiLevel =
|
||||
result.generationConfig?.thinkingConfig?.thinkingLevel ||
|
||||
result.generationConfig?.thinking_config?.thinkingLevel;
|
||||
if (typeof geminiLevel === "string" && THINKING_LEVEL_MAP[geminiLevel] !== undefined) {
|
||||
const budget = THINKING_LEVEL_MAP[geminiLevel];
|
||||
result.generationConfig = {
|
||||
...result.generationConfig,
|
||||
thinking_config: { thinking_budget: budget },
|
||||
};
|
||||
// Clean up camelCase variant if it was the source
|
||||
if (result.generationConfig.thinkingConfig) {
|
||||
delete result.generationConfig.thinkingConfig;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure models with -thinking suffix have thinking config injected.
|
||||
* Prevents 400 errors from Claude API when thinking params are missing.
|
||||
*
|
||||
* @param {object} body - Request body
|
||||
* @returns {object} Body with thinking config auto-injected if needed
|
||||
*/
|
||||
export function ensureThinkingConfig(body) {
|
||||
if (!body || typeof body !== "object") return body;
|
||||
const model = body.model || "";
|
||||
|
||||
// Only auto-inject for models with -thinking suffix
|
||||
if (!model.endsWith("-thinking")) return body;
|
||||
|
||||
// If thinking config already present, don't override
|
||||
if (body.thinking) return body;
|
||||
|
||||
const result = { ...body };
|
||||
result.thinking = {
|
||||
type: "enabled",
|
||||
budget_tokens: EFFORT_BUDGETS.medium, // 10240 default
|
||||
};
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply thinking budget control to a request body.
|
||||
* Called before format-specific translation.
|
||||
*
|
||||
* Pipeline: normalizeThinkingLevel → ensureThinkingConfig → mode processing
|
||||
*
|
||||
* @param {object} body - Request body (any format)
|
||||
* @param {object} [config] - Override config (defaults to stored config)
|
||||
* @returns {object} Modified body
|
||||
@@ -57,21 +137,27 @@ export function applyThinkingBudget(body, config = null) {
|
||||
const cfg = config || _config;
|
||||
if (!body || typeof body !== "object") return body;
|
||||
|
||||
// Pre-processing: convert string thinkingLevel to numeric budget
|
||||
let processed = normalizeThinkingLevel(body);
|
||||
|
||||
// Pre-processing: auto-inject thinking config for -thinking suffix models
|
||||
processed = ensureThinkingConfig(processed);
|
||||
|
||||
switch (cfg.mode) {
|
||||
case ThinkingMode.AUTO:
|
||||
return stripThinkingConfig(body);
|
||||
return stripThinkingConfig(processed);
|
||||
|
||||
case ThinkingMode.PASSTHROUGH:
|
||||
return body; // No changes
|
||||
return processed;
|
||||
|
||||
case ThinkingMode.CUSTOM:
|
||||
return setCustomBudget(body, cfg.customBudget);
|
||||
return setCustomBudget(processed, cfg.customBudget);
|
||||
|
||||
case ThinkingMode.ADAPTIVE:
|
||||
return applyAdaptiveBudget(body, cfg);
|
||||
return applyAdaptiveBudget(processed, cfg);
|
||||
|
||||
default:
|
||||
return body;
|
||||
return processed;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -151,9 +237,10 @@ function applyAdaptiveBudget(body, cfg) {
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
const msg = messages[i];
|
||||
if (msg.role === "user") {
|
||||
lastMsgLength = typeof msg.content === "string"
|
||||
? msg.content.length
|
||||
: JSON.stringify(msg.content || "").length;
|
||||
lastMsgLength =
|
||||
typeof msg.content === "string"
|
||||
? msg.content.length
|
||||
: JSON.stringify(msg.content || "").length;
|
||||
break;
|
||||
}
|
||||
}
|
||||
@@ -173,7 +260,7 @@ function applyAdaptiveBudget(body, cfg) {
|
||||
/**
|
||||
* Check if model name suggests thinking capability
|
||||
*/
|
||||
function hasThinkingCapableModel(body) {
|
||||
export function hasThinkingCapableModel(body) {
|
||||
const model = body.model || "";
|
||||
return (
|
||||
model.includes("claude") ||
|
||||
@@ -181,6 +268,7 @@ function hasThinkingCapableModel(body) {
|
||||
model.includes("o3") ||
|
||||
model.includes("o4") ||
|
||||
model.includes("gemini") ||
|
||||
model.endsWith("-thinking") ||
|
||||
model.includes("thinking")
|
||||
);
|
||||
}
|
||||
|
||||
@@ -323,9 +323,13 @@ export async function refreshQwenToken(refreshToken, log) {
|
||||
}
|
||||
|
||||
if (errorCode === "invalid_request") {
|
||||
log?.error?.("TOKEN_REFRESH", "Qwen refresh token is invalid or expired. Re-authentication required.", {
|
||||
status: response.status,
|
||||
});
|
||||
log?.error?.(
|
||||
"TOKEN_REFRESH",
|
||||
"Qwen refresh token is invalid or expired. Re-authentication required.",
|
||||
{
|
||||
status: response.status,
|
||||
}
|
||||
);
|
||||
return { error: "invalid_request" };
|
||||
}
|
||||
|
||||
@@ -720,8 +724,11 @@ export function supportsTokenRefresh(provider) {
|
||||
* Callers should stop retrying and request re-authentication.
|
||||
*/
|
||||
export function isUnrecoverableRefreshError(result) {
|
||||
return result && typeof result === "object" &&
|
||||
(result.error === "refresh_token_reused" || result.error === "invalid_request");
|
||||
return (
|
||||
result &&
|
||||
typeof result === "object" &&
|
||||
(result.error === "refresh_token_reused" || result.error === "invalid_request")
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -841,12 +848,103 @@ export async function getAllAccessTokens(userInfo, log) {
|
||||
/**
|
||||
* Refresh token with retry and exponential backoff
|
||||
* Retries on failure with increasing delay: 1s, 2s, 3s...
|
||||
*
|
||||
* Includes:
|
||||
* - Per-provider circuit breaker (5 consecutive failures → 30min pause)
|
||||
* - 30s timeout per refresh attempt to prevent hanging connections
|
||||
*
|
||||
* @param {function} refreshFn - Async function that returns token or null
|
||||
* @param {number} maxRetries - Max retry attempts (default 3)
|
||||
* @param {object} log - Logger instance (optional)
|
||||
* @param {string} provider - Provider ID for circuit breaker tracking (optional)
|
||||
* @returns {Promise<object|null>} Token result or null if all retries fail
|
||||
*/
|
||||
export async function refreshWithRetry(refreshFn, maxRetries = 3, log = null) {
|
||||
|
||||
// ─── Circuit Breaker State ──────────────────────────────────────────────────
|
||||
const _circuitBreaker: Record<string, { failures: number; blockedUntil: number }> = {};
|
||||
const CIRCUIT_BREAKER_THRESHOLD = 5; // consecutive failures before tripping
|
||||
const CIRCUIT_BREAKER_COOLDOWN = 30 * 60 * 1000; // 30 minutes
|
||||
const REFRESH_TIMEOUT_MS = 30_000; // 30s max per refresh attempt
|
||||
|
||||
/**
|
||||
* Check if a provider is circuit-breaker blocked.
|
||||
*/
|
||||
export function isProviderBlocked(provider: string): boolean {
|
||||
const state = _circuitBreaker[provider];
|
||||
if (!state) return false;
|
||||
if (state.blockedUntil > Date.now()) return true;
|
||||
// Cooldown expired — reset
|
||||
delete _circuitBreaker[provider];
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get circuit breaker status for all providers (for diagnostics).
|
||||
*/
|
||||
export function getCircuitBreakerStatus(): Record<string, any> {
|
||||
const result: Record<string, any> = {};
|
||||
for (const [provider, state] of Object.entries(_circuitBreaker)) {
|
||||
result[provider] = {
|
||||
failures: state.failures,
|
||||
blocked: state.blockedUntil > Date.now(),
|
||||
blockedUntil:
|
||||
state.blockedUntil > Date.now() ? new Date(state.blockedUntil).toISOString() : null,
|
||||
remainingMs: Math.max(0, state.blockedUntil - Date.now()),
|
||||
};
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a successful refresh — resets circuit breaker for provider.
|
||||
*/
|
||||
function recordSuccess(provider: string) {
|
||||
if (_circuitBreaker[provider]) {
|
||||
delete _circuitBreaker[provider];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a failed refresh — increments circuit breaker counter.
|
||||
*/
|
||||
function recordFailure(provider: string, log: any = null) {
|
||||
if (!_circuitBreaker[provider]) {
|
||||
_circuitBreaker[provider] = { failures: 0, blockedUntil: 0 };
|
||||
}
|
||||
_circuitBreaker[provider].failures++;
|
||||
|
||||
if (_circuitBreaker[provider].failures >= CIRCUIT_BREAKER_THRESHOLD) {
|
||||
_circuitBreaker[provider].blockedUntil = Date.now() + CIRCUIT_BREAKER_COOLDOWN;
|
||||
log?.error?.(
|
||||
"TOKEN_REFRESH",
|
||||
`🔴 Circuit breaker tripped for ${provider}: ${CIRCUIT_BREAKER_THRESHOLD} consecutive failures. ` +
|
||||
`Blocked for ${CIRCUIT_BREAKER_COOLDOWN / 60000}min. Provider needs re-authentication.`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a function with a timeout.
|
||||
*/
|
||||
async function withTimeout<T>(fn: () => Promise<T>, timeoutMs: number): Promise<T | null> {
|
||||
return Promise.race([
|
||||
fn(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), timeoutMs)),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function refreshWithRetry(
|
||||
refreshFn,
|
||||
maxRetries = 3,
|
||||
log = null,
|
||||
provider = "unknown"
|
||||
) {
|
||||
// Circuit breaker check
|
||||
if (isProviderBlocked(provider)) {
|
||||
log?.warn?.("TOKEN_REFRESH", `⚡ Circuit breaker active for ${provider}, skipping refresh`);
|
||||
return null;
|
||||
}
|
||||
|
||||
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
||||
if (attempt > 0) {
|
||||
const delay = attempt * 1000;
|
||||
@@ -855,13 +953,18 @@ export async function refreshWithRetry(refreshFn, maxRetries = 3, log = null) {
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await refreshFn();
|
||||
if (result) return result;
|
||||
const result = await withTimeout(refreshFn, REFRESH_TIMEOUT_MS);
|
||||
if (result) {
|
||||
recordSuccess(provider);
|
||||
return result;
|
||||
}
|
||||
} catch (error) {
|
||||
log?.warn?.("TOKEN_REFRESH", `Attempt ${attempt + 1}/${maxRetries} failed: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
log?.error?.("TOKEN_REFRESH", `All ${maxRetries} retry attempts failed`);
|
||||
// All retries exhausted — record failure for circuit breaker
|
||||
recordFailure(provider, log);
|
||||
log?.error?.("TOKEN_REFRESH", `All ${maxRetries} retry attempts failed for ${provider}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -180,7 +180,9 @@ function openaiToGeminiBase(model, body, stream) {
|
||||
functionDeclarations.push({
|
||||
name: t.name,
|
||||
description: t.description || "",
|
||||
parameters: t.input_schema || { type: "object", properties: {} },
|
||||
parameters: cleanJSONSchemaForAntigravity(
|
||||
t.input_schema || { type: "object", properties: {} }
|
||||
),
|
||||
});
|
||||
}
|
||||
// OpenAI format
|
||||
@@ -189,7 +191,9 @@ function openaiToGeminiBase(model, body, stream) {
|
||||
functionDeclarations.push({
|
||||
name: fn.name,
|
||||
description: fn.description || "",
|
||||
parameters: fn.parameters || { type: "object", properties: {} },
|
||||
parameters: cleanJSONSchemaForAntigravity(
|
||||
fn.parameters || { type: "object", properties: {} }
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -206,9 +210,7 @@ function openaiToGeminiBase(model, body, stream) {
|
||||
// Extract the schema (may be nested under .schema key)
|
||||
const schema = body.response_format.json_schema.schema || body.response_format.json_schema;
|
||||
if (schema && typeof schema === "object") {
|
||||
// Remove unsupported keywords for Gemini (it uses a subset of JSON Schema)
|
||||
const { $schema, additionalProperties, ...cleanSchema } = schema;
|
||||
result.generationConfig.responseSchema = cleanSchema;
|
||||
result.generationConfig.responseSchema = cleanJSONSchemaForAntigravity(schema);
|
||||
}
|
||||
} else if (body.response_format.type === "json_object") {
|
||||
result.generationConfig.responseMimeType = "application/json";
|
||||
@@ -269,8 +271,16 @@ export function openaiToGeminiCLIRequest(model, body, stream) {
|
||||
|
||||
// Wrap Gemini CLI format in Cloud Code wrapper
|
||||
function wrapInCloudCodeEnvelope(model, geminiCLI, credentials = null, isAntigravity = false) {
|
||||
const hasRealProject = !!credentials?.projectId;
|
||||
const projectId = credentials?.projectId || generateProjectId();
|
||||
|
||||
if (!hasRealProject) {
|
||||
console.warn(
|
||||
`[${isAntigravity ? "Antigravity" : "GeminiCLI"}] ⚠️ No projectId in credentials — using generated fallback "${projectId}". ` +
|
||||
`This may cause 404 errors. Ensure the OAuth token includes a valid GCP project.`
|
||||
);
|
||||
}
|
||||
|
||||
const cleanModel = model.includes("/") ? model.split("/").pop()! : model;
|
||||
|
||||
const envelope: Record<string, any> = {
|
||||
@@ -313,10 +323,17 @@ function wrapInCloudCodeEnvelope(model, geminiCLI, credentials = null, isAntigra
|
||||
return envelope;
|
||||
}
|
||||
|
||||
// Wrap Claude format in Cloud Code envelope for Antigravity
|
||||
function wrapInCloudCodeEnvelopeForClaude(model, claudeRequest, credentials = null) {
|
||||
const hasRealProject = !!credentials?.projectId;
|
||||
const projectId = credentials?.projectId || generateProjectId();
|
||||
|
||||
if (!hasRealProject) {
|
||||
console.warn(
|
||||
`[Antigravity/Claude] ⚠️ No projectId in credentials — using generated fallback "${projectId}". ` +
|
||||
`This may cause 404 errors. Ensure the OAuth token includes a valid GCP project.`
|
||||
);
|
||||
}
|
||||
|
||||
const cleanModel = model.includes("/") ? model.split("/").pop()! : model;
|
||||
|
||||
const envelope: Record<string, any> = {
|
||||
|
||||
@@ -364,6 +364,7 @@ export function openaiResponsesToOpenAIResponse(chunk, state) {
|
||||
// Flush: send final chunk with finish_reason
|
||||
if (!state.finishReasonSent && state.started) {
|
||||
state.finishReasonSent = true;
|
||||
const hadToolCalls = (state.toolCallIndex || 0) > 0;
|
||||
return {
|
||||
id: state.chatId || `chatcmpl-${Date.now()}`,
|
||||
object: "chat.completion.chunk",
|
||||
@@ -373,7 +374,7 @@ export function openaiResponsesToOpenAIResponse(chunk, state) {
|
||||
{
|
||||
index: 0,
|
||||
delta: {},
|
||||
finish_reason: "stop",
|
||||
finish_reason: hadToolCalls ? "tool_calls" : "stop",
|
||||
},
|
||||
],
|
||||
};
|
||||
@@ -468,6 +469,8 @@ export function openaiResponsesToOpenAIResponse(chunk, state) {
|
||||
tool_calls: [
|
||||
{
|
||||
index: state.toolCallIndex,
|
||||
id: state.currentToolCallId,
|
||||
type: "function",
|
||||
function: { arguments: argsDelta },
|
||||
},
|
||||
],
|
||||
@@ -517,7 +520,9 @@ export function openaiResponsesToOpenAIResponse(chunk, state) {
|
||||
|
||||
if (!state.finishReasonSent) {
|
||||
state.finishReasonSent = true;
|
||||
state.finishReason = "stop"; // Mark for usage injection in stream.js
|
||||
const hadToolCalls = (state.toolCallIndex || 0) > 0;
|
||||
const reason = hadToolCalls ? "tool_calls" : "stop";
|
||||
state.finishReason = reason; // Mark for usage injection in stream.js
|
||||
|
||||
const finalChunk: Record<string, any> = {
|
||||
id: state.chatId,
|
||||
@@ -528,7 +533,7 @@ export function openaiResponsesToOpenAIResponse(chunk, state) {
|
||||
{
|
||||
index: 0,
|
||||
delta: {},
|
||||
finish_reason: "stop",
|
||||
finish_reason: reason,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
/**
|
||||
* Shared ComfyUI API Client
|
||||
*
|
||||
* Used by image, video, and music handlers to submit workflows,
|
||||
* poll for completion, and fetch output files from a ComfyUI server.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Submit a workflow to ComfyUI for execution.
|
||||
* @returns The prompt_id for polling
|
||||
*/
|
||||
export async function submitComfyWorkflow(
|
||||
baseUrl: string,
|
||||
workflow: object
|
||||
): Promise<string> {
|
||||
const res = await fetch(`${baseUrl}/prompt`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ prompt: workflow }),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
throw new Error(`ComfyUI submit failed (${res.status}): ${errText}`);
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
return data.prompt_id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Poll ComfyUI history endpoint until the prompt completes or times out.
|
||||
* @returns The history entry for the completed prompt
|
||||
*/
|
||||
export async function pollComfyResult(
|
||||
baseUrl: string,
|
||||
promptId: string,
|
||||
timeoutMs: number = 120_000
|
||||
): Promise<any> {
|
||||
const start = Date.now();
|
||||
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
await new Promise((r) => setTimeout(r, 2000));
|
||||
|
||||
const res = await fetch(`${baseUrl}/history/${promptId}`);
|
||||
if (!res.ok) continue;
|
||||
|
||||
const data = await res.json();
|
||||
const entry = data[promptId];
|
||||
|
||||
if (entry && entry.outputs && Object.keys(entry.outputs).length > 0) {
|
||||
return entry;
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`ComfyUI prompt ${promptId} timed out after ${timeoutMs}ms`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch an output file from ComfyUI.
|
||||
* @returns The file contents as ArrayBuffer
|
||||
*/
|
||||
export async function fetchComfyOutput(
|
||||
baseUrl: string,
|
||||
filename: string,
|
||||
subfolder: string,
|
||||
type: string
|
||||
): Promise<ArrayBuffer> {
|
||||
const url = new URL(`${baseUrl}/view`);
|
||||
url.searchParams.set("filename", filename);
|
||||
url.searchParams.set("subfolder", subfolder);
|
||||
url.searchParams.set("type", type);
|
||||
|
||||
const res = await fetch(url.toString());
|
||||
if (!res.ok) {
|
||||
throw new Error(`ComfyUI fetch output failed (${res.status})`);
|
||||
}
|
||||
|
||||
return res.arrayBuffer();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract output files from a ComfyUI history entry.
|
||||
* Returns an array of { filename, subfolder, type } for each output.
|
||||
*/
|
||||
export function extractComfyOutputFiles(
|
||||
historyEntry: any
|
||||
): Array<{ filename: string; subfolder: string; type: string }> {
|
||||
const files: Array<{ filename: string; subfolder: string; type: string }> = [];
|
||||
|
||||
for (const nodeOutput of Object.values(historyEntry.outputs || {})) {
|
||||
const outputs = (nodeOutput as any).images || (nodeOutput as any).gifs || (nodeOutput as any).audio || [];
|
||||
for (const file of outputs) {
|
||||
files.push({
|
||||
filename: file.filename,
|
||||
subfolder: file.subfolder || "",
|
||||
type: file.type || "output",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return files;
|
||||
}
|
||||
@@ -4,13 +4,39 @@ import { socksDispatcher } from "fetch-socks";
|
||||
const DISPATCHER_CACHE_KEY = Symbol.for("omniroute.proxyDispatcher.cache");
|
||||
const SUPPORTED_PROTOCOLS = new Set(["http:", "https:", "socks5:"]);
|
||||
|
||||
function getDispatcherCache() {
|
||||
function getDispatcherCache(): Map<string, any> {
|
||||
if (!globalThis[DISPATCHER_CACHE_KEY]) {
|
||||
globalThis[DISPATCHER_CACHE_KEY] = new Map();
|
||||
}
|
||||
return globalThis[DISPATCHER_CACHE_KEY];
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all cached proxy dispatchers.
|
||||
* Call this when proxy configuration changes to avoid stale connections.
|
||||
*/
|
||||
export function clearDispatcherCache() {
|
||||
const cache = getDispatcherCache();
|
||||
cache.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the port from a proxy URL string before URL parsing.
|
||||
* `new URL("http://host:80")` strips port 80 since it's the HTTP default,
|
||||
* but proxy servers commonly listen on port 80/443, so we need to preserve it.
|
||||
*/
|
||||
function extractExplicitPort(urlStr) {
|
||||
try {
|
||||
// Match port in the host portion: "scheme://[user:pass@]host:PORT[/...]"
|
||||
const match = urlStr.match(/:\/\/(?:[^@]*@)?[^:/\s]+:(\d+)/);
|
||||
if (match) {
|
||||
const port = Number(match[1]);
|
||||
if (Number.isInteger(port) && port >= 1 && port <= 65535) return String(port);
|
||||
}
|
||||
} catch {}
|
||||
return null;
|
||||
}
|
||||
|
||||
function defaultPortForProtocol(protocol) {
|
||||
if (protocol === "https:" || protocol === "wss:") return "443";
|
||||
if (protocol === "socks5:") return "1080";
|
||||
@@ -26,13 +52,28 @@ function normalizePort(port, protocol) {
|
||||
return String(parsed);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a proxy URL string manually from parsed URL components.
|
||||
* We cannot use URL.toString() because the URL serializer silently strips
|
||||
* default ports (80 for http, 443 for https). Proxy servers commonly
|
||||
* listen on these ports, so we must always include the port explicitly.
|
||||
*/
|
||||
function buildProxyUrlString(parsed, port) {
|
||||
const auth =
|
||||
parsed.username
|
||||
? `${parsed.username}${parsed.password ? `:${parsed.password}` : ""}@`
|
||||
: "";
|
||||
return `${parsed.protocol}//${auth}${parsed.hostname}:${port}`;
|
||||
}
|
||||
|
||||
export function isSocks5ProxyEnabled() {
|
||||
return process.env.ENABLE_SOCKS5_PROXY === "true";
|
||||
}
|
||||
|
||||
export function proxyUrlForLogs(proxyUrl) {
|
||||
const explicitPort = extractExplicitPort(proxyUrl);
|
||||
const parsed = new URL(proxyUrl);
|
||||
const port = parsed.port || defaultPortForProtocol(parsed.protocol);
|
||||
const port = explicitPort || parsed.port || defaultPortForProtocol(parsed.protocol);
|
||||
return `${parsed.protocol}//${parsed.hostname}:${port}`;
|
||||
}
|
||||
|
||||
@@ -41,6 +82,11 @@ export function normalizeProxyUrl(
|
||||
source = "proxy",
|
||||
{ allowSocks5 = isSocks5ProxyEnabled() } = {}
|
||||
) {
|
||||
// Extract the explicit port from the raw URL string BEFORE parsing,
|
||||
// because `new URL()` silently strips default ports (80 for http,
|
||||
// 443 for https), which are valid and common for proxy servers.
|
||||
const explicitPort = extractExplicitPort(proxyUrl);
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = new URL(proxyUrl);
|
||||
@@ -62,8 +108,12 @@ export function normalizeProxyUrl(
|
||||
throw new Error(`[ProxyDispatcher] Invalid ${source} host`);
|
||||
}
|
||||
|
||||
parsed.port = normalizePort(parsed.port, parsed.protocol);
|
||||
return parsed.toString();
|
||||
// Use the explicit port from the raw string if present, otherwise apply default.
|
||||
const port = explicitPort || normalizePort(parsed.port, parsed.protocol);
|
||||
|
||||
// Build the URL string manually instead of using parsed.toString(),
|
||||
// which would strip default ports (80/443) and break the proxy connection.
|
||||
return buildProxyUrlString(parsed, port);
|
||||
}
|
||||
|
||||
export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabled() } = {}) {
|
||||
@@ -93,14 +143,15 @@ export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabl
|
||||
}
|
||||
|
||||
const port = normalizePort(proxyConfig.port, protocol);
|
||||
const proxyUrl = new URL(`${type}://${proxyConfig.host}:${port}`);
|
||||
|
||||
if (proxyConfig.username) {
|
||||
proxyUrl.username = proxyConfig.username;
|
||||
proxyUrl.password = proxyConfig.password || "";
|
||||
}
|
||||
// Build the URL string manually to preserve the port through normalization.
|
||||
const auth = proxyConfig.username
|
||||
? `${encodeURIComponent(proxyConfig.username)}:${proxyConfig.password ? encodeURIComponent(proxyConfig.password) : ""}@`
|
||||
: "";
|
||||
|
||||
return normalizeProxyUrl(proxyUrl.toString(), "context proxy", { allowSocks5 });
|
||||
const proxyUrlStr = `${type}://${auth}${proxyConfig.host}:${port}`;
|
||||
|
||||
return normalizeProxyUrl(proxyUrlStr, "context proxy", { allowSocks5 });
|
||||
}
|
||||
|
||||
export function createProxyDispatcher(proxyUrl) {
|
||||
@@ -111,11 +162,14 @@ export function createProxyDispatcher(proxyUrl) {
|
||||
if (dispatcher) return dispatcher;
|
||||
|
||||
const parsed = new URL(normalizedUrl);
|
||||
const explicitPort = extractExplicitPort(normalizedUrl);
|
||||
const port = explicitPort || normalizePort(parsed.port, parsed.protocol);
|
||||
|
||||
if (parsed.protocol === "socks5:") {
|
||||
const socksOptions: Record<string, any> = {
|
||||
type: 5,
|
||||
host: parsed.hostname,
|
||||
port: Number(normalizePort(parsed.port, parsed.protocol)),
|
||||
port: Number(port),
|
||||
};
|
||||
if (parsed.username) socksOptions.userId = decodeURIComponent(parsed.username);
|
||||
if (parsed.password) socksOptions.password = decodeURIComponent(parsed.password);
|
||||
@@ -126,4 +180,4 @@ export function createProxyDispatcher(proxyUrl) {
|
||||
|
||||
dispatcherCache.set(normalizedUrl, dispatcher);
|
||||
return dispatcher;
|
||||
}
|
||||
}
|
||||
Generated
+24
-41
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "1.6.4",
|
||||
"version": "1.7.12",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "omniroute",
|
||||
"version": "1.6.4",
|
||||
"version": "1.7.12",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
@@ -3063,9 +3063,9 @@
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/node": {
|
||||
"version": "25.3.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.3.0.tgz",
|
||||
"integrity": "sha512-4K3bqJpXpqfg2XKGK9bpDTc6xO/xoUP/RBWS7AtRMug6zZFaRekiLzjVtAoZMquxoAbzBvy5nxQ7veS5eYzf8A==",
|
||||
"version": "25.3.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.3.3.tgz",
|
||||
"integrity": "sha512-DpzbrH7wIcBaJibpKo9nnSQL0MTRdnWttGyE5haGwK86xgMOkFLp7vEyfQPGLOJh5wNYiJ3V9PmUMDhV9u8kkQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"undici-types": "~7.18.0"
|
||||
@@ -7763,19 +7763,18 @@
|
||||
}
|
||||
},
|
||||
"node_modules/lint-staged": {
|
||||
"version": "16.2.7",
|
||||
"resolved": "https://registry.npmjs.org/lint-staged/-/lint-staged-16.2.7.tgz",
|
||||
"integrity": "sha512-lDIj4RnYmK7/kXMya+qJsmkRFkGolciXjrsZ6PC25GdTfWOAWetR0ZbsNXRAj1EHHImRSalc+whZFg56F5DVow==",
|
||||
"version": "16.3.1",
|
||||
"resolved": "https://registry.npmjs.org/lint-staged/-/lint-staged-16.3.1.tgz",
|
||||
"integrity": "sha512-bqvvquXzFBAlSbluugR4KXAe4XnO/QZcKVszpkBtqLWa2KEiVy8n6Xp38OeUbv/gOJOX4Vo9u5pFt/ADvbm42Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"commander": "^14.0.2",
|
||||
"commander": "^14.0.3",
|
||||
"listr2": "^9.0.5",
|
||||
"micromatch": "^4.0.8",
|
||||
"nano-spawn": "^2.0.0",
|
||||
"pidtree": "^0.6.0",
|
||||
"string-argv": "^0.3.2",
|
||||
"yaml": "^2.8.1"
|
||||
"tinyexec": "^1.0.2",
|
||||
"yaml": "^2.8.2"
|
||||
},
|
||||
"bin": {
|
||||
"lint-staged": "bin/lint-staged.js"
|
||||
@@ -8084,19 +8083,6 @@
|
||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/nano-spawn": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/nano-spawn/-/nano-spawn-2.0.0.tgz",
|
||||
"integrity": "sha512-tacvGzUY5o2D8CBh2rrwxyNojUsZNU2zjNTzKQrkgGJQTbGAfArVWXSKMBokBeeg6C7OLRGUEyoFlYbfeWQIqw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=20.17"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sindresorhus/nano-spawn?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/nanoid": {
|
||||
"version": "3.3.11",
|
||||
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz",
|
||||
@@ -8730,19 +8716,6 @@
|
||||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/pidtree": {
|
||||
"version": "0.6.0",
|
||||
"resolved": "https://registry.npmjs.org/pidtree/-/pidtree-0.6.0.tgz",
|
||||
"integrity": "sha512-eG2dWTVw5bzqGRztnHExczNxt5VGsE6OwTeCG3fdUf9KBsZzO3R5OIIIzWR+iZA0NtZ+RDVdaoE2dK1cn6jH4g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"bin": {
|
||||
"pidtree": "bin/pidtree.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.10"
|
||||
}
|
||||
},
|
||||
"node_modules/pino": {
|
||||
"version": "10.3.1",
|
||||
"resolved": "https://registry.npmjs.org/pino/-/pino-10.3.1.tgz",
|
||||
@@ -10384,6 +10357,16 @@
|
||||
"integrity": "sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/tinyexec": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-1.0.2.tgz",
|
||||
"integrity": "sha512-W/KYk+NFhkmsYpuHq5JykngiOCnxeVL8v8dFnqxSD8qEEdRfXk1SDM6JzNqcERbcGYj9tMrDQBYV9cjgnunFIg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/tinyglobby": {
|
||||
"version": "0.2.15",
|
||||
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz",
|
||||
@@ -11100,9 +11083,9 @@
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/wreq-js": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/wreq-js/-/wreq-js-2.0.1.tgz",
|
||||
"integrity": "sha512-7GcZpzVtRX/A5pqu2QXTcqKTiYJZZRIRiosJ94GT1+taQivysZmjvkwzk8IvLz1oE1P1rGWG8ujDLCYxusM1aQ==",
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/wreq-js/-/wreq-js-2.1.1.tgz",
|
||||
"integrity": "sha512-nJBOMBTczqcyHpF8a8YdPyxb30htK2RxuAfr6O8a6oyKHj2nRPjXbZcGXrquIdZx1b+6NV/GHweD3OqWwE7n4A==",
|
||||
"cpu": [
|
||||
"x64",
|
||||
"arm64"
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "1.6.8",
|
||||
"version": "1.7.12",
|
||||
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 441 KiB |
@@ -33,7 +33,6 @@ export default function CLIToolsPageClient({ machineId }) {
|
||||
const [apiKeys, setApiKeys] = useState([]);
|
||||
const [toolStatuses, setToolStatuses] = useState({});
|
||||
const [statusesLoaded, setStatusesLoaded] = useState(false);
|
||||
const [apiBaseUrl, setApiBaseUrl] = useState("");
|
||||
|
||||
useEffect(() => {
|
||||
fetchConnections();
|
||||
@@ -48,12 +47,6 @@ export default function CLIToolsPageClient({ machineId }) {
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
setCloudEnabled(data.cloudEnabled || false);
|
||||
if (typeof window !== "undefined") {
|
||||
const protocol = window.location.protocol;
|
||||
const hostname = window.location.hostname;
|
||||
const apiPort = data?.apiPort || 20128;
|
||||
setApiBaseUrl(`${protocol}//${hostname}:${apiPort}`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.log("Error loading cloud settings:", error);
|
||||
@@ -155,9 +148,8 @@ export default function CLIToolsPageClient({ machineId }) {
|
||||
if (cloudEnabled && CLOUD_URL) {
|
||||
return CLOUD_URL;
|
||||
}
|
||||
if (apiBaseUrl) {
|
||||
return apiBaseUrl;
|
||||
}
|
||||
// Use window.location.origin directly — works correctly in Docker/reverse-proxy
|
||||
// Per @alpgul feedback: don't use baseUrl prop (has port duplication issues)
|
||||
if (typeof window !== "undefined") {
|
||||
return window.location.origin;
|
||||
}
|
||||
|
||||
@@ -60,7 +60,7 @@ export default function CopilotToolCard({
|
||||
.then((data) => {
|
||||
if (cancelled) return;
|
||||
const modelList = (data.data || [])
|
||||
.filter((m: any) => m.id) // Only models with valid IDs
|
||||
.filter((m: any) => m && !m.type && !m.parent && m.id) // Only chat models with valid IDs
|
||||
.map((m: any) => ({
|
||||
value: m.id,
|
||||
label: m.id,
|
||||
@@ -112,11 +112,8 @@ export default function CopilotToolCard({
|
||||
};
|
||||
|
||||
const getBaseUrlForConfig = () => {
|
||||
// Use window.location.origin directly — works correctly in Docker/reverse-proxy
|
||||
// Per @alpgul feedback: don't use baseUrl prop (has port duplication issues)
|
||||
const origin =
|
||||
typeof window !== "undefined" ? window.location.origin : "http://localhost:20128";
|
||||
return `${origin}/v1/chat/completions`;
|
||||
const url = baseUrl;
|
||||
return `${url}/v1/chat/completions`;
|
||||
};
|
||||
|
||||
// Generate the Copilot chatLanguageModels.json config
|
||||
|
||||
@@ -7,7 +7,7 @@ import { useCopyToClipboard } from "@/shared/hooks/useCopyToClipboard";
|
||||
import { AI_PROVIDERS, getProviderByAlias } from "@/shared/constants/providers";
|
||||
import { useTranslations } from "next-intl";
|
||||
|
||||
const CLOUD_URL = process.env.NEXT_PUBLIC_CLOUD_URL;
|
||||
const CLOUD_URL = process.env.NEXT_PUBLIC_CLOUD_URL || null;
|
||||
const CLOUD_ACTION_TIMEOUT_MS = 15000;
|
||||
|
||||
export default function APIPageClient({ machineId }) {
|
||||
@@ -29,6 +29,7 @@ export default function APIPageClient({ machineId }) {
|
||||
const [syncStep, setSyncStep] = useState(""); // "syncing" | "verifying" | "disabling" | "done" | ""
|
||||
const [modalSuccess, setModalSuccess] = useState(false); // show success state in modal before closing
|
||||
const [selectedProvider, setSelectedProvider] = useState(null); // for provider models popup
|
||||
const [cloudBaseUrl, setCloudBaseUrl] = useState(CLOUD_URL); // dynamic cloud URL from API response
|
||||
|
||||
const { copied, copy } = useCopyToClipboard();
|
||||
|
||||
@@ -204,6 +205,10 @@ export default function APIPageClient({ machineId }) {
|
||||
if (data.createdKey) {
|
||||
await fetchData();
|
||||
}
|
||||
// Update cloud URL from API response (fixes undefined/v1 when env var not set)
|
||||
if (data.cloudUrl) {
|
||||
setCloudBaseUrl(data.cloudUrl);
|
||||
}
|
||||
// Reload settings to ensure fresh state
|
||||
await loadCloudSettings();
|
||||
} else {
|
||||
@@ -274,7 +279,7 @@ export default function APIPageClient({ machineId }) {
|
||||
};
|
||||
|
||||
const [baseUrl, setBaseUrl] = useState("/v1");
|
||||
const cloudEndpointNew = `${CLOUD_URL}/v1`;
|
||||
const cloudEndpointNew = cloudBaseUrl ? `${cloudBaseUrl}/v1` : null;
|
||||
|
||||
// Hydration fix: Only access window on client side
|
||||
useEffect(() => {
|
||||
@@ -293,7 +298,7 @@ export default function APIPageClient({ machineId }) {
|
||||
}
|
||||
|
||||
// Use new format endpoint (machineId embedded in key)
|
||||
const currentEndpoint = cloudEnabled ? cloudEndpointNew : baseUrl;
|
||||
const currentEndpoint = cloudEnabled && cloudEndpointNew ? cloudEndpointNew : baseUrl;
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-8">
|
||||
@@ -382,7 +387,7 @@ export default function APIPageClient({ machineId }) {
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
{/* Available Endpoints */}
|
||||
{/* Available Endpoints */}
|
||||
<Card>
|
||||
<div className="flex items-center justify-between mb-4">
|
||||
<div>
|
||||
@@ -398,134 +403,200 @@ export default function APIPageClient({ machineId }) {
|
||||
endpointData.audioTranscription,
|
||||
endpointData.audioSpeech,
|
||||
endpointData.moderation,
|
||||
].filter((a) => a.length > 0).length,
|
||||
].filter((a) => a.length > 0).length + 2,
|
||||
})}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex flex-col gap-3">
|
||||
{/* Chat Completions */}
|
||||
<EndpointSection
|
||||
icon="chat"
|
||||
iconColor="text-blue-500"
|
||||
iconBg="bg-blue-500/10"
|
||||
title={t("chatCompletions")}
|
||||
path="/v1/chat/completions"
|
||||
description={t("chatDesc")}
|
||||
models={endpointData.chat}
|
||||
expanded={expandedEndpoint === "chat"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "chat" ? null : "chat")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Core APIs */}
|
||||
<div className="mb-6">
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<span className="material-symbols-outlined text-sm text-primary">hub</span>
|
||||
<h3 className="text-xs font-semibold text-text-muted uppercase tracking-wider">
|
||||
{t("categoryCore") || "Core APIs"}
|
||||
</h3>
|
||||
<div className="flex-1 h-px bg-border/50" />
|
||||
</div>
|
||||
<div className="flex flex-col gap-3">
|
||||
{/* Chat Completions */}
|
||||
<EndpointSection
|
||||
icon="chat"
|
||||
iconColor="text-blue-500"
|
||||
iconBg="bg-blue-500/10"
|
||||
title={t("chatCompletions")}
|
||||
path="/v1/chat/completions"
|
||||
description={t("chatDesc")}
|
||||
models={endpointData.chat}
|
||||
expanded={expandedEndpoint === "chat"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "chat" ? null : "chat")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
|
||||
{/* Embeddings */}
|
||||
<EndpointSection
|
||||
icon="data_array"
|
||||
iconColor="text-emerald-500"
|
||||
iconBg="bg-emerald-500/10"
|
||||
title={t("embeddings")}
|
||||
path="/v1/embeddings"
|
||||
description={t("embeddingsDesc")}
|
||||
models={endpointData.embeddings}
|
||||
expanded={expandedEndpoint === "embeddings"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(expandedEndpoint === "embeddings" ? null : "embeddings")
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Responses API */}
|
||||
<EndpointSection
|
||||
icon="code"
|
||||
iconColor="text-indigo-500"
|
||||
iconBg="bg-indigo-500/10"
|
||||
title={t("responses") || "Responses API"}
|
||||
path="/v1/responses"
|
||||
description={t("responsesDesc") || "OpenAI Responses API for Codex and advanced agentic workflows"}
|
||||
models={endpointData.chat}
|
||||
expanded={expandedEndpoint === "responses"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "responses" ? null : "responses")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Image Generation */}
|
||||
<EndpointSection
|
||||
icon="image"
|
||||
iconColor="text-purple-500"
|
||||
iconBg="bg-purple-500/10"
|
||||
title={t("imageGeneration")}
|
||||
path="/v1/images/generations"
|
||||
description={t("imageDesc")}
|
||||
models={endpointData.images}
|
||||
expanded={expandedEndpoint === "images"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "images" ? null : "images")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Media & Multi-Modal */}
|
||||
<div className="mb-6">
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<span className="material-symbols-outlined text-sm text-purple-400">perm_media</span>
|
||||
<h3 className="text-xs font-semibold text-text-muted uppercase tracking-wider">
|
||||
{t("categoryMedia") || "Media & Multi-Modal"}
|
||||
</h3>
|
||||
<div className="flex-1 h-px bg-border/50" />
|
||||
</div>
|
||||
<div className="flex flex-col gap-3">
|
||||
{/* Embeddings */}
|
||||
<EndpointSection
|
||||
icon="data_array"
|
||||
iconColor="text-emerald-500"
|
||||
iconBg="bg-emerald-500/10"
|
||||
title={t("embeddings")}
|
||||
path="/v1/embeddings"
|
||||
description={t("embeddingsDesc")}
|
||||
models={endpointData.embeddings}
|
||||
expanded={expandedEndpoint === "embeddings"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(expandedEndpoint === "embeddings" ? null : "embeddings")
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
|
||||
{/* Rerank */}
|
||||
<EndpointSection
|
||||
icon="sort"
|
||||
iconColor="text-amber-500"
|
||||
iconBg="bg-amber-500/10"
|
||||
title={t("rerank")}
|
||||
path="/v1/rerank"
|
||||
description={t("rerankDesc")}
|
||||
models={endpointData.rerank}
|
||||
expanded={expandedEndpoint === "rerank"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "rerank" ? null : "rerank")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Image Generation */}
|
||||
<EndpointSection
|
||||
icon="image"
|
||||
iconColor="text-purple-500"
|
||||
iconBg="bg-purple-500/10"
|
||||
title={t("imageGeneration")}
|
||||
path="/v1/images/generations"
|
||||
description={t("imageDesc")}
|
||||
models={endpointData.images}
|
||||
expanded={expandedEndpoint === "images"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "images" ? null : "images")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
|
||||
{/* Audio Transcription */}
|
||||
<EndpointSection
|
||||
icon="mic"
|
||||
iconColor="text-rose-500"
|
||||
iconBg="bg-rose-500/10"
|
||||
title={t("audioTranscription")}
|
||||
path="/v1/audio/transcriptions"
|
||||
description={t("audioTranscriptionDesc")}
|
||||
models={endpointData.audioTranscription}
|
||||
expanded={expandedEndpoint === "audioTranscription"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(
|
||||
expandedEndpoint === "audioTranscription" ? null : "audioTranscription"
|
||||
)
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Audio Transcription */}
|
||||
<EndpointSection
|
||||
icon="mic"
|
||||
iconColor="text-rose-500"
|
||||
iconBg="bg-rose-500/10"
|
||||
title={t("audioTranscription")}
|
||||
path="/v1/audio/transcriptions"
|
||||
description={t("audioTranscriptionDesc")}
|
||||
models={endpointData.audioTranscription}
|
||||
expanded={expandedEndpoint === "audioTranscription"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(
|
||||
expandedEndpoint === "audioTranscription" ? null : "audioTranscription"
|
||||
)
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
|
||||
{/* Audio Speech (TTS) */}
|
||||
<EndpointSection
|
||||
icon="record_voice_over"
|
||||
iconColor="text-cyan-500"
|
||||
iconBg="bg-cyan-500/10"
|
||||
title={t("textToSpeech")}
|
||||
path="/v1/audio/speech"
|
||||
description={t("textToSpeechDesc")}
|
||||
models={endpointData.audioSpeech}
|
||||
expanded={expandedEndpoint === "audioSpeech"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(expandedEndpoint === "audioSpeech" ? null : "audioSpeech")
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Audio Speech (TTS) */}
|
||||
<EndpointSection
|
||||
icon="record_voice_over"
|
||||
iconColor="text-cyan-500"
|
||||
iconBg="bg-cyan-500/10"
|
||||
title={t("textToSpeech")}
|
||||
path="/v1/audio/speech"
|
||||
description={t("textToSpeechDesc")}
|
||||
models={endpointData.audioSpeech}
|
||||
expanded={expandedEndpoint === "audioSpeech"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(expandedEndpoint === "audioSpeech" ? null : "audioSpeech")
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Moderations */}
|
||||
<EndpointSection
|
||||
icon="shield"
|
||||
iconColor="text-orange-500"
|
||||
iconBg="bg-orange-500/10"
|
||||
title={t("moderations")}
|
||||
path="/v1/moderations"
|
||||
description={t("moderationsDesc")}
|
||||
models={endpointData.moderation}
|
||||
expanded={expandedEndpoint === "moderation"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(expandedEndpoint === "moderation" ? null : "moderation")
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
{/* Utility & Management */}
|
||||
<div>
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<span className="material-symbols-outlined text-sm text-amber-400">build</span>
|
||||
<h3 className="text-xs font-semibold text-text-muted uppercase tracking-wider">
|
||||
{t("categoryUtility") || "Utility & Management"}
|
||||
</h3>
|
||||
<div className="flex-1 h-px bg-border/50" />
|
||||
</div>
|
||||
<div className="flex flex-col gap-3">
|
||||
{/* Rerank */}
|
||||
<EndpointSection
|
||||
icon="sort"
|
||||
iconColor="text-amber-500"
|
||||
iconBg="bg-amber-500/10"
|
||||
title={t("rerank")}
|
||||
path="/v1/rerank"
|
||||
description={t("rerankDesc")}
|
||||
models={endpointData.rerank}
|
||||
expanded={expandedEndpoint === "rerank"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "rerank" ? null : "rerank")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
|
||||
{/* Moderations */}
|
||||
<EndpointSection
|
||||
icon="shield"
|
||||
iconColor="text-orange-500"
|
||||
iconBg="bg-orange-500/10"
|
||||
title={t("moderations")}
|
||||
path="/v1/moderations"
|
||||
description={t("moderationsDesc")}
|
||||
models={endpointData.moderation}
|
||||
expanded={expandedEndpoint === "moderation"}
|
||||
onToggle={() =>
|
||||
setExpandedEndpoint(expandedEndpoint === "moderation" ? null : "moderation")
|
||||
}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
|
||||
{/* List Models */}
|
||||
<EndpointSection
|
||||
icon="list"
|
||||
iconColor="text-teal-500"
|
||||
iconBg="bg-teal-500/10"
|
||||
title={t("listModels") || "List Models"}
|
||||
path="/v1/models"
|
||||
description={t("listModelsDesc") || "List all available models across all connected providers"}
|
||||
models={[]}
|
||||
expanded={expandedEndpoint === "models"}
|
||||
onToggle={() => setExpandedEndpoint(expandedEndpoint === "models" ? null : "models")}
|
||||
copy={copy}
|
||||
copied={copied}
|
||||
baseUrl={currentEndpoint}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { useTranslations } from "next-intl";
|
||||
|
||||
type Modality = "image" | "video" | "music";
|
||||
type GenerationResult = {
|
||||
type: Modality;
|
||||
data: any;
|
||||
timestamp: number;
|
||||
};
|
||||
|
||||
const MODALITY_CONFIG: Record<
|
||||
Modality,
|
||||
{ icon: string; endpoint: string; label: string; placeholder: string; color: string }
|
||||
> = {
|
||||
image: {
|
||||
icon: "image",
|
||||
endpoint: "/api/v1/images/generations",
|
||||
label: "Image Generation",
|
||||
placeholder: "A serene landscape with mountains at sunset...",
|
||||
color: "from-purple-500 to-pink-500",
|
||||
},
|
||||
video: {
|
||||
icon: "videocam",
|
||||
endpoint: "/api/v1/videos/generations",
|
||||
label: "Video Generation",
|
||||
placeholder: "A timelapse of a flower blooming...",
|
||||
color: "from-blue-500 to-cyan-500",
|
||||
},
|
||||
music: {
|
||||
icon: "music_note",
|
||||
endpoint: "/api/v1/music/generations",
|
||||
label: "Music Generation",
|
||||
placeholder: "Upbeat electronic music with synth pads...",
|
||||
color: "from-orange-500 to-yellow-500",
|
||||
},
|
||||
};
|
||||
|
||||
export default function MediaPageClient() {
|
||||
const t = useTranslations("media");
|
||||
const [activeTab, setActiveTab] = useState<Modality>("image");
|
||||
const [prompt, setPrompt] = useState("");
|
||||
const [model, setModel] = useState("");
|
||||
const [models, setModels] = useState<any[]>([]);
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [loadingModels, setLoadingModels] = useState(false);
|
||||
const [result, setResult] = useState<GenerationResult | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
// Fetch available models for each modality
|
||||
const fetchModels = async (modality: Modality) => {
|
||||
setLoadingModels(true);
|
||||
try {
|
||||
const res = await fetch(MODALITY_CONFIG[modality].endpoint);
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
const modelList = data.data || [];
|
||||
setModels(modelList);
|
||||
if (modelList.length > 0) setModel(modelList[0].id);
|
||||
}
|
||||
} catch {
|
||||
setModels([]);
|
||||
}
|
||||
setLoadingModels(false);
|
||||
};
|
||||
|
||||
const switchTab = (tab: Modality) => {
|
||||
setActiveTab(tab);
|
||||
setPrompt("");
|
||||
setResult(null);
|
||||
setError(null);
|
||||
fetchModels(tab);
|
||||
};
|
||||
|
||||
const handleGenerate = async () => {
|
||||
if (!prompt.trim()) return;
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
setResult(null);
|
||||
|
||||
try {
|
||||
const config = MODALITY_CONFIG[activeTab];
|
||||
const res = await fetch(config.endpoint, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model: model || undefined,
|
||||
prompt: prompt.trim(),
|
||||
...(activeTab === "image" ? { size: "1024x1024", n: 1 } : {}),
|
||||
}),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errData = await res.json().catch(() => ({}));
|
||||
throw new Error(errData?.error?.message || `Generation failed (${res.status})`);
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
setResult({ type: activeTab, data, timestamp: Date.now() });
|
||||
} catch (err: any) {
|
||||
setError(err.message || "Generation failed");
|
||||
}
|
||||
setLoading(false);
|
||||
};
|
||||
|
||||
// Load models on first render
|
||||
useState(() => {
|
||||
fetchModels("image");
|
||||
});
|
||||
|
||||
const config = MODALITY_CONFIG[activeTab];
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold text-text-main">{t("title")}</h1>
|
||||
<p className="text-text-muted text-sm mt-1">{t("subtitle")}</p>
|
||||
</div>
|
||||
|
||||
{/* Modality Tabs */}
|
||||
<div className="flex gap-2 p-1 bg-surface/50 rounded-xl border border-black/5 dark:border-white/5">
|
||||
{(Object.keys(MODALITY_CONFIG) as Modality[]).map((key) => {
|
||||
const cfg = MODALITY_CONFIG[key];
|
||||
const isActive = key === activeTab;
|
||||
return (
|
||||
<button
|
||||
key={key}
|
||||
onClick={() => switchTab(key)}
|
||||
className={`flex-1 flex items-center justify-center gap-2 px-4 py-2.5 rounded-lg text-sm font-medium transition-all ${
|
||||
isActive
|
||||
? "bg-primary/10 text-primary shadow-sm border border-primary/20"
|
||||
: "text-text-muted hover:text-text-main hover:bg-surface/80"
|
||||
}`}
|
||||
>
|
||||
<span className="material-symbols-outlined text-[18px]">{cfg.icon}</span>
|
||||
{cfg.label}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
|
||||
{/* Generation Form */}
|
||||
<div className="bg-surface/30 rounded-xl border border-black/5 dark:border-white/5 p-6 space-y-4">
|
||||
{/* Model selector */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium text-text-main mb-2">{t("model")}</label>
|
||||
{loadingModels ? (
|
||||
<div className="flex items-center gap-2 text-text-muted text-sm">
|
||||
<span className="material-symbols-outlined animate-spin text-[16px]">progress_activity</span>
|
||||
{t("loadingModels")}
|
||||
</div>
|
||||
) : models.length > 0 ? (
|
||||
<select
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
className="w-full px-3 py-2 rounded-lg bg-surface border border-black/10 dark:border-white/10 text-text-main text-sm focus:outline-none focus:ring-2 focus:ring-primary/30"
|
||||
>
|
||||
{models.map((m) => (
|
||||
<option key={m.id} value={m.id}>
|
||||
{m.id}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
) : (
|
||||
<p className="text-text-muted text-sm">{t("noModels")}</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Prompt */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium text-text-main mb-2">{t("prompt")}</label>
|
||||
<textarea
|
||||
rows={3}
|
||||
value={prompt}
|
||||
onChange={(e) => setPrompt(e.target.value)}
|
||||
placeholder={config.placeholder}
|
||||
className="w-full px-3 py-2 rounded-lg bg-surface border border-black/10 dark:border-white/10 text-text-main text-sm placeholder:text-text-muted/50 focus:outline-none focus:ring-2 focus:ring-primary/30 resize-none"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Generate button */}
|
||||
<button
|
||||
onClick={handleGenerate}
|
||||
disabled={loading || !prompt.trim()}
|
||||
className={`w-full flex items-center justify-center gap-2 px-4 py-3 rounded-lg text-white font-medium transition-all bg-gradient-to-r ${config.color} ${
|
||||
loading || !prompt.trim() ? "opacity-50 cursor-not-allowed" : "hover:opacity-90 hover:shadow-lg"
|
||||
}`}
|
||||
>
|
||||
{loading ? (
|
||||
<>
|
||||
<span className="material-symbols-outlined animate-spin text-[18px]">progress_activity</span>
|
||||
{t("generating")}
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<span className="material-symbols-outlined text-[18px]">auto_awesome</span>
|
||||
{t("generate")} {config.label}
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Error */}
|
||||
{error && (
|
||||
<div className="bg-red-500/10 border border-red-500/20 rounded-xl p-4 flex items-start gap-3">
|
||||
<span className="material-symbols-outlined text-red-500 text-[20px] mt-0.5">error</span>
|
||||
<div>
|
||||
<p className="text-sm font-medium text-red-500">{t("error")}</p>
|
||||
<p className="text-sm text-text-muted mt-1">{error}</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Result */}
|
||||
{result && (
|
||||
<div className="bg-surface/30 rounded-xl border border-black/5 dark:border-white/5 p-6">
|
||||
<div className="flex items-center gap-2 mb-4">
|
||||
<span className={`material-symbols-outlined text-[20px] bg-gradient-to-r ${config.color} bg-clip-text text-transparent`}>
|
||||
{config.icon}
|
||||
</span>
|
||||
<h3 className="text-sm font-medium text-text-main">{t("result")}</h3>
|
||||
<span className="text-xs text-text-muted ml-auto">
|
||||
{new Date(result.timestamp).toLocaleTimeString()}
|
||||
</span>
|
||||
</div>
|
||||
<pre className="bg-surface rounded-lg p-4 text-xs text-text-muted overflow-auto max-h-96 custom-scrollbar">
|
||||
{JSON.stringify(result.data, null, 2)}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Info cards */}
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
|
||||
{(Object.keys(MODALITY_CONFIG) as Modality[]).map((key) => {
|
||||
const cfg = MODALITY_CONFIG[key];
|
||||
return (
|
||||
<div
|
||||
key={key}
|
||||
className="bg-surface/30 rounded-xl border border-black/5 dark:border-white/5 p-4"
|
||||
>
|
||||
<div className="flex items-center gap-2 mb-2">
|
||||
<div className={`flex items-center justify-center size-8 rounded-lg bg-gradient-to-r ${cfg.color}`}>
|
||||
<span className="material-symbols-outlined text-white text-[16px]">{cfg.icon}</span>
|
||||
</div>
|
||||
<span className="text-sm font-medium text-text-main">{cfg.label}</span>
|
||||
</div>
|
||||
<p className="text-xs text-text-muted">
|
||||
{t(`${key}Description`)}
|
||||
</p>
|
||||
<code className="block mt-2 text-xs text-primary/70 bg-primary/5 rounded px-2 py-1">
|
||||
POST {cfg.endpoint}
|
||||
</code>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import MediaPageClient from "./MediaPageClient";
|
||||
|
||||
export default function MediaPage() {
|
||||
return <MediaPageClient />;
|
||||
}
|
||||
@@ -1,16 +1,30 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useEffect } from "react";
|
||||
import { Card, Toggle } from "@/shared/components";
|
||||
import { Button, Card, Toggle } from "@/shared/components";
|
||||
import { useTheme } from "@/shared/hooks/useTheme";
|
||||
import useThemeStore, { COLOR_THEMES } from "@/store/themeStore";
|
||||
import { cn } from "@/shared/utils/cn";
|
||||
import { useTranslations } from "next-intl";
|
||||
|
||||
export default function AppearanceTab() {
|
||||
const { theme, setTheme, isDark } = useTheme();
|
||||
const { colorTheme, customColor, setColorTheme, setCustomColorTheme } = useThemeStore();
|
||||
const t = useTranslations("settings");
|
||||
const [settings, setSettings] = useState<Record<string, any>>({});
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [customThemeColor, setCustomThemeColor] = useState(customColor || "#3b82f6");
|
||||
const isValidHex = /^#([0-9a-fA-F]{6})$/.test(customThemeColor.startsWith("#") ? customThemeColor : `#${customThemeColor}`);
|
||||
|
||||
// Subscribe to store changes (e.g. from another tab) via Zustand external subscription
|
||||
useEffect(() => {
|
||||
const unsubscribe = useThemeStore.subscribe((state) => {
|
||||
if (state.customColor && state.customColor !== customThemeColor) {
|
||||
setCustomThemeColor(state.customColor);
|
||||
}
|
||||
});
|
||||
return unsubscribe;
|
||||
}, []); // eslint-disable-line react-hooks/exhaustive-deps
|
||||
const themeOptionLabels: Record<string, string> = {
|
||||
light: t("themeLight"),
|
||||
dark: t("themeDark"),
|
||||
@@ -47,6 +61,16 @@ export default function AppearanceTab() {
|
||||
}
|
||||
};
|
||||
|
||||
const presetThemes = [
|
||||
{ id: "coral", color: COLOR_THEMES.coral, label: t("themeCoral") },
|
||||
{ id: "blue", color: COLOR_THEMES.blue, label: t("themeBlue") },
|
||||
{ id: "red", color: COLOR_THEMES.red, label: t("themeRed") },
|
||||
{ id: "green", color: COLOR_THEMES.green, label: t("themeGreen") },
|
||||
{ id: "violet", color: COLOR_THEMES.violet, label: t("themeViolet") },
|
||||
{ id: "orange", color: COLOR_THEMES.orange, label: t("themeOrange") },
|
||||
{ id: "cyan", color: COLOR_THEMES.cyan, label: t("themeCyan") },
|
||||
];
|
||||
|
||||
return (
|
||||
<Card>
|
||||
<div className="flex items-center gap-3 mb-4">
|
||||
@@ -94,6 +118,56 @@ export default function AppearanceTab() {
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="pt-4 border-t border-border">
|
||||
<p className="font-medium mb-1">{t("themeAccent")}</p>
|
||||
<p className="text-sm text-text-muted mb-3">{t("themeAccentDesc")}</p>
|
||||
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-2 mb-3">
|
||||
{presetThemes.map((item) => {
|
||||
const active = colorTheme === item.id;
|
||||
return (
|
||||
<button
|
||||
key={item.id}
|
||||
onClick={() => setColorTheme(item.id)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 p-2 rounded-lg border transition-colors",
|
||||
active
|
||||
? "border-primary bg-primary/10 text-primary"
|
||||
: "border-border hover:bg-surface/50 text-text-main"
|
||||
)}
|
||||
>
|
||||
<span className="flex items-center gap-2">
|
||||
<span
|
||||
className="size-4 rounded-full border border-black/10 dark:border-white/20"
|
||||
style={{ backgroundColor: item.color }}
|
||||
/>
|
||||
<span className="text-sm font-medium">{item.label}</span>
|
||||
</span>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-2">
|
||||
<input
|
||||
type="color"
|
||||
value={customThemeColor}
|
||||
onChange={(e) => setCustomThemeColor(e.target.value)}
|
||||
className="h-10 w-12 rounded border border-border bg-surface cursor-pointer"
|
||||
aria-label={t("themeCustom")}
|
||||
/>
|
||||
<input
|
||||
type="text"
|
||||
value={customThemeColor}
|
||||
onChange={(e) => setCustomThemeColor(e.target.value)}
|
||||
placeholder="#3b82f6"
|
||||
maxLength={7}
|
||||
className={`flex-1 h-10 px-3 rounded-lg bg-surface border text-sm text-text-main focus:outline-none ${isValidHex ? "border-border focus:border-primary" : "border-red-400 focus:border-red-500"}`}
|
||||
/>
|
||||
<Button onClick={() => setCustomColorTheme(customThemeColor)} disabled={!isValidHex}>{t("themeCreate")}</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="pt-4 border-t border-border">
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useEffect } from "react";
|
||||
import { Card } from "@/shared/components";
|
||||
import { useTranslations } from "next-intl";
|
||||
|
||||
export default function BackgroundDegradationTab() {
|
||||
const [config, setConfig] = useState({
|
||||
enabled: false,
|
||||
degradationMap: {},
|
||||
detectionPatterns: [],
|
||||
stats: { detected: 0, tokensSaved: 0 },
|
||||
});
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [status, setStatus] = useState("");
|
||||
const [newFrom, setNewFrom] = useState("");
|
||||
const [newTo, setNewTo] = useState("");
|
||||
const [newPattern, setNewPattern] = useState("");
|
||||
const t = useTranslations("settings");
|
||||
|
||||
useEffect(() => {
|
||||
fetch("/api/settings/background-degradation")
|
||||
.then((res) => res.json())
|
||||
.then((data) => {
|
||||
setConfig(data);
|
||||
setLoading(false);
|
||||
})
|
||||
.catch(() => setLoading(false));
|
||||
}, []);
|
||||
|
||||
const save = async (updates) => {
|
||||
const newConfig = { ...config, ...updates };
|
||||
setConfig(newConfig);
|
||||
setSaving(true);
|
||||
setStatus("");
|
||||
try {
|
||||
const { stats, ...persistable } = newConfig;
|
||||
const res = await fetch("/api/settings/background-degradation", {
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(persistable),
|
||||
});
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
setConfig(data);
|
||||
setStatus("saved");
|
||||
setTimeout(() => setStatus(""), 2000);
|
||||
} else {
|
||||
setStatus("error");
|
||||
}
|
||||
} catch {
|
||||
setStatus("error");
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const addMapping = () => {
|
||||
if (!newFrom.trim() || !newTo.trim()) return;
|
||||
const map = { ...config.degradationMap, [newFrom.trim()]: newTo.trim() };
|
||||
save({ degradationMap: map });
|
||||
setNewFrom("");
|
||||
setNewTo("");
|
||||
};
|
||||
|
||||
const removeMapping = (key) => {
|
||||
const map = { ...config.degradationMap };
|
||||
delete map[key];
|
||||
save({ degradationMap: map });
|
||||
};
|
||||
|
||||
const addPattern = () => {
|
||||
if (!newPattern.trim()) return;
|
||||
const patterns = [...config.detectionPatterns, newPattern.trim()];
|
||||
save({ detectionPatterns: patterns });
|
||||
setNewPattern("");
|
||||
};
|
||||
|
||||
const removePattern = (idx) => {
|
||||
const patterns = config.detectionPatterns.filter((_, i) => i !== idx);
|
||||
save({ detectionPatterns: patterns });
|
||||
};
|
||||
|
||||
const mapEntries = Object.entries(config.degradationMap || {}) as [string, string][];
|
||||
|
||||
return (
|
||||
<Card>
|
||||
<div className="flex items-center gap-3 mb-5">
|
||||
<div className="p-2 rounded-lg bg-sky-500/10 text-sky-500">
|
||||
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
|
||||
speed
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex-1">
|
||||
<h3 className="text-lg font-semibold">
|
||||
{t("backgroundDegradationTitle") || "Background Task Degradation"}
|
||||
</h3>
|
||||
<p className="text-sm text-text-muted">
|
||||
{t("backgroundDegradationDesc") ||
|
||||
"Auto-redirect background requests (titles, summaries) to cheaper models"}
|
||||
</p>
|
||||
</div>
|
||||
{status === "saved" && (
|
||||
<span className="text-xs font-medium text-emerald-500 flex items-center gap-1">
|
||||
<span className="material-symbols-outlined text-[14px]">check_circle</span>{" "}
|
||||
{t("saved") || "Saved"}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Toggle */}
|
||||
<div className="flex items-center justify-between p-4 rounded-lg bg-surface/30 border border-border/30 mb-4">
|
||||
<div>
|
||||
<p className="text-sm font-medium">
|
||||
{t("enableDegradation") || "Enable Background Degradation"}
|
||||
</p>
|
||||
<p className="text-xs text-text-muted mt-0.5">
|
||||
{t("enableDegradationHint") ||
|
||||
"Automatically use cheaper models for background utility tasks"}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
onClick={() => save({ enabled: !config.enabled })}
|
||||
disabled={loading || saving}
|
||||
className={`relative inline-flex h-6 w-11 items-center rounded-full transition-colors ${
|
||||
config.enabled ? "bg-sky-500" : "bg-white/10"
|
||||
}`}
|
||||
>
|
||||
<span
|
||||
className={`inline-block h-4 w-4 rounded-full bg-white transition-transform ${
|
||||
config.enabled ? "translate-x-6" : "translate-x-1"
|
||||
}`}
|
||||
/>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Stats */}
|
||||
{config.stats && config.stats.detected > 0 && (
|
||||
<div className="flex items-center gap-4 p-3 rounded-lg bg-sky-500/5 border border-sky-500/20 mb-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="material-symbols-outlined text-[16px] text-sky-400">analytics</span>
|
||||
<span className="text-xs text-text-muted">
|
||||
{t("tasksDetected") || "Tasks detected"}:
|
||||
</span>
|
||||
<span className="text-sm font-mono font-semibold text-sky-400">
|
||||
{config.stats.detected}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{config.enabled && (
|
||||
<>
|
||||
{/* Degradation Map */}
|
||||
<div className="mb-4">
|
||||
<p className="text-xs font-medium text-text-muted uppercase tracking-wider mb-2">
|
||||
{t("degradationMap") || "Model Degradation Map"}
|
||||
</p>
|
||||
|
||||
{/* Add new mapping */}
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<input
|
||||
type="text"
|
||||
placeholder={t("premiumModel") || "Premium model"}
|
||||
value={newFrom}
|
||||
onChange={(e) => setNewFrom(e.target.value)}
|
||||
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-sky-500/50 focus:outline-none"
|
||||
/>
|
||||
<span className="text-text-muted text-lg">→</span>
|
||||
<input
|
||||
type="text"
|
||||
placeholder={t("cheapModel") || "Cheap model"}
|
||||
value={newTo}
|
||||
onChange={(e) => setNewTo(e.target.value)}
|
||||
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-sky-500/50 focus:outline-none"
|
||||
/>
|
||||
<button
|
||||
onClick={addMapping}
|
||||
disabled={saving || !newFrom.trim() || !newTo.trim()}
|
||||
className="px-3 py-2 rounded-lg text-sm font-medium bg-sky-500/10 text-sky-500 hover:bg-sky-500/20 disabled:opacity-50 transition-all"
|
||||
>
|
||||
{t("add") || "Add"}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Existing mappings */}
|
||||
{mapEntries.length > 0 && (
|
||||
<div className="rounded-lg border border-border/30 divide-y divide-border/20 max-h-48 overflow-y-auto">
|
||||
{mapEntries.map(([from, to]) => (
|
||||
<div key={from} className="flex items-center gap-3 px-4 py-2">
|
||||
<code className="text-xs text-orange-400/80 flex-1 truncate">{from}</code>
|
||||
<span className="material-symbols-outlined text-[14px] text-text-muted">
|
||||
arrow_forward
|
||||
</span>
|
||||
<code className="text-xs text-sky-400/80 flex-1 truncate">{to}</code>
|
||||
<button
|
||||
onClick={() => removeMapping(from)}
|
||||
disabled={saving}
|
||||
className="p-1 rounded hover:bg-red-500/10 text-text-muted hover:text-red-400 transition-all"
|
||||
>
|
||||
<span className="material-symbols-outlined text-[16px]">close</span>
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Detection Patterns */}
|
||||
<details className="group">
|
||||
<summary className="text-xs font-medium text-text-muted uppercase tracking-wider cursor-pointer flex items-center gap-1 mb-2">
|
||||
<span className="material-symbols-outlined text-[14px] group-open:rotate-90 transition-transform">
|
||||
chevron_right
|
||||
</span>
|
||||
{t("detectionPatterns") || "Detection Patterns"} (
|
||||
{config.detectionPatterns?.length || 0})
|
||||
</summary>
|
||||
|
||||
{/* Add new pattern */}
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<input
|
||||
type="text"
|
||||
placeholder={t("newPattern") || 'e.g. "generate a title"'}
|
||||
value={newPattern}
|
||||
onChange={(e) => setNewPattern(e.target.value)}
|
||||
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-sky-500/50 focus:outline-none"
|
||||
/>
|
||||
<button
|
||||
onClick={addPattern}
|
||||
disabled={saving || !newPattern.trim()}
|
||||
className="px-3 py-2 rounded-lg text-sm font-medium bg-sky-500/10 text-sky-500 hover:bg-sky-500/20 disabled:opacity-50 transition-all"
|
||||
>
|
||||
{t("add") || "Add"}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Existing patterns */}
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{(config.detectionPatterns || []).map((pattern, idx) => (
|
||||
<span
|
||||
key={idx}
|
||||
className="inline-flex items-center gap-1 px-2.5 py-1 rounded-full text-xs bg-sky-500/10 text-sky-400 border border-sky-500/20"
|
||||
>
|
||||
{pattern}
|
||||
<button
|
||||
onClick={() => removePattern(idx)}
|
||||
className="hover:text-red-400 transition-colors"
|
||||
disabled={saving}
|
||||
>
|
||||
<span className="material-symbols-outlined text-[12px]">close</span>
|
||||
</button>
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
</details>
|
||||
</>
|
||||
)}
|
||||
</Card>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useEffect } from "react";
|
||||
import { Card } from "@/shared/components";
|
||||
import { useTranslations } from "next-intl";
|
||||
|
||||
export default function ModelAliasesTab() {
|
||||
const [builtIn, setBuiltIn] = useState({});
|
||||
const [custom, setCustom] = useState({});
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [status, setStatus] = useState("");
|
||||
const [newFrom, setNewFrom] = useState("");
|
||||
const [newTo, setNewTo] = useState("");
|
||||
const t = useTranslations("settings");
|
||||
|
||||
useEffect(() => {
|
||||
fetch("/api/settings/model-aliases")
|
||||
.then((res) => res.json())
|
||||
.then((data) => {
|
||||
setBuiltIn(data.builtIn || {});
|
||||
setCustom(data.custom || {});
|
||||
setLoading(false);
|
||||
})
|
||||
.catch(() => setLoading(false));
|
||||
}, []);
|
||||
|
||||
const addAlias = async () => {
|
||||
if (!newFrom.trim() || !newTo.trim()) return;
|
||||
setSaving(true);
|
||||
try {
|
||||
const res = await fetch("/api/settings/model-aliases", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ from: newFrom.trim(), to: newTo.trim() }),
|
||||
});
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
setCustom(data.custom);
|
||||
setNewFrom("");
|
||||
setNewTo("");
|
||||
setStatus("saved");
|
||||
setTimeout(() => setStatus(""), 2000);
|
||||
}
|
||||
} catch {
|
||||
setStatus("error");
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const removeAlias = async (from) => {
|
||||
setSaving(true);
|
||||
try {
|
||||
const res = await fetch("/api/settings/model-aliases", {
|
||||
method: "DELETE",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ from }),
|
||||
});
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
setCustom(data.custom);
|
||||
setStatus("saved");
|
||||
setTimeout(() => setStatus(""), 2000);
|
||||
}
|
||||
} catch {
|
||||
setStatus("error");
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const builtInEntries = Object.entries(builtIn);
|
||||
const customEntries = Object.entries(custom);
|
||||
|
||||
return (
|
||||
<Card>
|
||||
<div className="flex items-center gap-3 mb-5">
|
||||
<div className="p-2 rounded-lg bg-amber-500/10 text-amber-500">
|
||||
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
|
||||
swap_horiz
|
||||
</span>
|
||||
</div>
|
||||
<div>
|
||||
<h3 className="text-lg font-semibold">{t("modelAliasesTitle") || "Model Aliases"}</h3>
|
||||
<p className="text-sm text-text-muted">
|
||||
{t("modelAliasesDesc") || "Auto-forward deprecated model IDs to their replacements"}
|
||||
</p>
|
||||
</div>
|
||||
{status === "saved" && (
|
||||
<span className="ml-auto text-xs font-medium text-emerald-500 flex items-center gap-1">
|
||||
<span className="material-symbols-outlined text-[14px]">check_circle</span>{" "}
|
||||
{t("saved") || "Saved"}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Add custom alias */}
|
||||
<div className="p-4 rounded-lg bg-surface/30 border border-border/30 mb-4">
|
||||
<p className="text-sm font-medium mb-3">
|
||||
{t("addCustomAlias") || "Add Custom Alias"}
|
||||
</p>
|
||||
<div className="flex items-center gap-2">
|
||||
<input
|
||||
type="text"
|
||||
placeholder={t("deprecatedModelId") || "Deprecated model ID"}
|
||||
value={newFrom}
|
||||
onChange={(e) => setNewFrom(e.target.value)}
|
||||
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-amber-500/50 focus:outline-none"
|
||||
/>
|
||||
<span className="text-text-muted text-lg">→</span>
|
||||
<input
|
||||
type="text"
|
||||
placeholder={t("newModelId") || "New model ID"}
|
||||
value={newTo}
|
||||
onChange={(e) => setNewTo(e.target.value)}
|
||||
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-amber-500/50 focus:outline-none"
|
||||
/>
|
||||
<button
|
||||
onClick={addAlias}
|
||||
disabled={saving || !newFrom.trim() || !newTo.trim()}
|
||||
className="px-4 py-2 rounded-lg text-sm font-medium bg-amber-500/10 text-amber-500 hover:bg-amber-500/20 disabled:opacity-50 transition-all"
|
||||
>
|
||||
{t("add") || "Add"}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Custom aliases */}
|
||||
{customEntries.length > 0 && (
|
||||
<div className="mb-4">
|
||||
<p className="text-xs font-medium text-text-muted uppercase tracking-wider mb-2">
|
||||
{t("customAliases") || "Custom Aliases"}
|
||||
</p>
|
||||
<div className="rounded-lg border border-border/30 divide-y divide-border/20">
|
||||
{customEntries.map(([from, to]) => (
|
||||
<div key={from} className="flex items-center gap-3 px-4 py-2.5">
|
||||
<code className="text-xs text-red-400/80 flex-1 truncate">{from}</code>
|
||||
<span className="material-symbols-outlined text-[14px] text-text-muted">
|
||||
arrow_forward
|
||||
</span>
|
||||
<code className="text-xs text-emerald-400/80 flex-1 truncate">{to}</code>
|
||||
<button
|
||||
onClick={() => removeAlias(from)}
|
||||
disabled={saving}
|
||||
className="p-1 rounded hover:bg-red-500/10 text-text-muted hover:text-red-400 transition-all"
|
||||
>
|
||||
<span className="material-symbols-outlined text-[16px]">close</span>
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Built-in aliases (collapsed by default) */}
|
||||
<details className="group">
|
||||
<summary className="text-xs font-medium text-text-muted uppercase tracking-wider cursor-pointer flex items-center gap-1 mb-2">
|
||||
<span className="material-symbols-outlined text-[14px] group-open:rotate-90 transition-transform">
|
||||
chevron_right
|
||||
</span>
|
||||
{t("builtInAliases") || "Built-in Aliases"} ({builtInEntries.length})
|
||||
</summary>
|
||||
<div className="rounded-lg border border-border/30 divide-y divide-border/20 max-h-60 overflow-y-auto">
|
||||
{builtInEntries.map(([from, to]) => (
|
||||
<div key={from} className="flex items-center gap-3 px-4 py-2 opacity-60">
|
||||
<code className="text-xs text-red-400/60 flex-1 truncate">{from}</code>
|
||||
<span className="material-symbols-outlined text-[14px] text-text-muted">
|
||||
arrow_forward
|
||||
</span>
|
||||
<code className="text-xs text-emerald-400/60 flex-1 truncate">{to}</code>
|
||||
<span className="material-symbols-outlined text-[14px] text-text-muted">lock</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</details>
|
||||
</Card>
|
||||
);
|
||||
}
|
||||
@@ -13,6 +13,8 @@ import ProxyTab from "./components/ProxyTab";
|
||||
import AppearanceTab from "./components/AppearanceTab";
|
||||
import ThinkingBudgetTab from "./components/ThinkingBudgetTab";
|
||||
import SystemPromptTab from "./components/SystemPromptTab";
|
||||
import ModelAliasesTab from "./components/ModelAliasesTab";
|
||||
import BackgroundDegradationTab from "./components/BackgroundDegradationTab";
|
||||
|
||||
import CacheStatsCard from "./components/CacheStatsCard";
|
||||
import ResilienceTab from "./components/ResilienceTab";
|
||||
@@ -94,6 +96,8 @@ export default function SettingsPage() {
|
||||
<div className="flex flex-col gap-6">
|
||||
<RoutingTab />
|
||||
<ComboDefaultsTab />
|
||||
<ModelAliasesTab />
|
||||
<BackgroundDegradationTab />
|
||||
</div>
|
||||
)}
|
||||
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
import os from "os";
|
||||
import { ensureCliConfigWriteAllowed, getCliConfigPaths } from "@/shared/services/cliRuntime";
|
||||
import { resolveDataDir } from "@/lib/dataPaths";
|
||||
|
||||
const PROFILES_DIR = path.join(os.homedir(), ".omniroute", "codex-profiles");
|
||||
const PROFILES_DIR = path.join(resolveDataDir(), "codex-profiles");
|
||||
|
||||
/**
|
||||
* Ensure profiles directory exists
|
||||
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
requestDeviceCode,
|
||||
pollForToken,
|
||||
} from "@/lib/oauth/providers";
|
||||
import { createProviderConnection, isCloudEnabled } from "@/models";
|
||||
import { createProviderConnection, updateProviderConnection, getProviderConnections, isCloudEnabled } from "@/models";
|
||||
import { getConsistentMachineId } from "@/shared/utils/machineId";
|
||||
import { syncToCloud } from "@/lib/cloudSync";
|
||||
import { startLocalServer } from "@/lib/oauth/utils/server";
|
||||
@@ -170,16 +170,33 @@ export async function POST(
|
||||
exchangeTokens(provider, code, redirectUri, codeVerifier, state)
|
||||
);
|
||||
|
||||
// Save to database
|
||||
const connection: any = await createProviderConnection({
|
||||
provider,
|
||||
authType: "oauth",
|
||||
...tokenData,
|
||||
expiresAt: tokenData.expiresIn
|
||||
? new Date(Date.now() + tokenData.expiresIn * 1000).toISOString()
|
||||
: null,
|
||||
testStatus: "active",
|
||||
});
|
||||
// Upsert: update existing connection if same provider+email, else create new
|
||||
const expiresAt = tokenData.expiresIn
|
||||
? new Date(Date.now() + tokenData.expiresIn * 1000).toISOString()
|
||||
: null;
|
||||
|
||||
let connection: any;
|
||||
if (tokenData.email) {
|
||||
const existing = await getProviderConnections({ provider });
|
||||
const match = existing.find((c: any) => c.email === tokenData.email && c.authType === "oauth");
|
||||
if (match) {
|
||||
connection = await updateProviderConnection(match.id, {
|
||||
...tokenData,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
isActive: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!connection) {
|
||||
connection = await createProviderConnection({
|
||||
provider,
|
||||
authType: "oauth",
|
||||
...tokenData,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
});
|
||||
}
|
||||
|
||||
// Auto sync to Cloud if enabled
|
||||
await syncToCloudIfEnabled();
|
||||
@@ -218,16 +235,33 @@ export async function POST(
|
||||
}
|
||||
|
||||
if (result.success) {
|
||||
// Save to database
|
||||
const connection: any = await createProviderConnection({
|
||||
provider,
|
||||
authType: "oauth",
|
||||
...result.tokens,
|
||||
expiresAt: result.tokens.expiresIn
|
||||
? new Date(Date.now() + result.tokens.expiresIn * 1000).toISOString()
|
||||
: null,
|
||||
testStatus: "active",
|
||||
});
|
||||
// Upsert: update existing connection if same provider+email, else create new
|
||||
const expiresAt = result.tokens.expiresIn
|
||||
? new Date(Date.now() + result.tokens.expiresIn * 1000).toISOString()
|
||||
: null;
|
||||
|
||||
let connection: any;
|
||||
if (result.tokens.email) {
|
||||
const existing = await getProviderConnections({ provider });
|
||||
const match = existing.find((c: any) => c.email === result.tokens.email && c.authType === "oauth");
|
||||
if (match) {
|
||||
connection = await updateProviderConnection(match.id, {
|
||||
...result.tokens,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
isActive: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!connection) {
|
||||
connection = await createProviderConnection({
|
||||
provider,
|
||||
authType: "oauth",
|
||||
...result.tokens,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
});
|
||||
}
|
||||
|
||||
// Auto sync to Cloud if enabled
|
||||
await syncToCloudIfEnabled();
|
||||
@@ -312,16 +346,33 @@ export async function POST(
|
||||
exchangeTokens(provider, params.code, redirectUri, codeVerifier, params.state)
|
||||
);
|
||||
|
||||
// Save to database
|
||||
const connection: any = await createProviderConnection({
|
||||
provider,
|
||||
authType: "oauth",
|
||||
...tokenData,
|
||||
expiresAt: tokenData.expiresIn
|
||||
? new Date(Date.now() + tokenData.expiresIn * 1000).toISOString()
|
||||
: null,
|
||||
testStatus: "active",
|
||||
});
|
||||
// Upsert: update existing connection if same provider+email, else create new
|
||||
const expiresAt = tokenData.expiresIn
|
||||
? new Date(Date.now() + tokenData.expiresIn * 1000).toISOString()
|
||||
: null;
|
||||
|
||||
let connection: any;
|
||||
if (tokenData.email) {
|
||||
const existing = await getProviderConnections({ provider });
|
||||
const match = existing.find((c: any) => c.email === tokenData.email && c.authType === "oauth");
|
||||
if (match) {
|
||||
connection = await updateProviderConnection(match.id, {
|
||||
...tokenData,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
isActive: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!connection) {
|
||||
connection = await createProviderConnection({
|
||||
provider,
|
||||
authType: "oauth",
|
||||
...tokenData,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
});
|
||||
}
|
||||
|
||||
await syncToCloudIfEnabled();
|
||||
|
||||
|
||||
@@ -49,7 +49,12 @@ const PROVIDER_MODELS_CONFIG = {
|
||||
method: "GET",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
authQuery: "key", // Use query param for API key
|
||||
parseResponse: (data) => data.models || [],
|
||||
parseResponse: (data) =>
|
||||
(data.models || []).map((m) => ({
|
||||
...m,
|
||||
id: (m.name || m.id || "").replace(/^models\//, ""),
|
||||
name: m.displayName || (m.name || "").replace(/^models\//, ""),
|
||||
})),
|
||||
},
|
||||
"gemini-cli": {
|
||||
url: "https://generativelanguage.googleapis.com/v1beta/models",
|
||||
@@ -57,7 +62,12 @@ const PROVIDER_MODELS_CONFIG = {
|
||||
headers: { "Content-Type": "application/json" },
|
||||
authHeader: "Authorization",
|
||||
authPrefix: "Bearer ",
|
||||
parseResponse: (data) => data.models || [],
|
||||
parseResponse: (data) =>
|
||||
(data.models || []).map((m) => ({
|
||||
...m,
|
||||
id: (m.name || m.id || "").replace(/^models\//, ""),
|
||||
name: m.displayName || (m.name || "").replace(/^models\//, ""),
|
||||
})),
|
||||
},
|
||||
qwen: {
|
||||
url: "https://portal.qwen.ai/v1/models",
|
||||
@@ -133,6 +143,14 @@ const PROVIDER_MODELS_CONFIG = {
|
||||
authPrefix: "Bearer ",
|
||||
parseResponse: (data) => data.data || data.models || [],
|
||||
},
|
||||
blackbox: {
|
||||
url: "https://api.blackbox.ai/v1/models",
|
||||
method: "GET",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
authHeader: "Authorization",
|
||||
authPrefix: "Bearer ",
|
||||
parseResponse: (data) => data.data || data.models || [],
|
||||
},
|
||||
xai: {
|
||||
url: "https://api.x.ai/v1/models",
|
||||
method: "GET",
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import {
|
||||
getBackgroundDegradationConfig,
|
||||
setBackgroundDegradationConfig,
|
||||
resetStats,
|
||||
} from "@omniroute/open-sse/services/backgroundTaskDetector.ts";
|
||||
import { updateSettings } from "@/lib/db/settings";
|
||||
|
||||
/**
|
||||
* GET /api/settings/background-degradation
|
||||
* Returns the current background degradation configuration.
|
||||
*/
|
||||
export async function GET() {
|
||||
try {
|
||||
return NextResponse.json(getBackgroundDegradationConfig());
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/background-degradation GET:", error);
|
||||
return NextResponse.json({ error: "Failed to get config" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* PUT /api/settings/background-degradation
|
||||
* Update the background degradation configuration.
|
||||
* Body: { enabled?: boolean, degradationMap?: {...}, detectionPatterns?: [...] }
|
||||
*/
|
||||
export async function PUT(request) {
|
||||
try {
|
||||
const config = await request.json();
|
||||
setBackgroundDegradationConfig(config);
|
||||
|
||||
// Persist to database (excluding stats)
|
||||
const { stats, ...persistable } = getBackgroundDegradationConfig();
|
||||
await updateSettings({ backgroundDegradation: JSON.stringify(persistable) });
|
||||
|
||||
return NextResponse.json({ success: true, ...getBackgroundDegradationConfig() });
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/background-degradation PUT:", error);
|
||||
return NextResponse.json({ error: "Failed to update config" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/settings/background-degradation
|
||||
* Reset stats counters.
|
||||
* Body: { action: "reset-stats" }
|
||||
*/
|
||||
export async function POST(request) {
|
||||
try {
|
||||
const { action } = await request.json();
|
||||
if (action === "reset-stats") {
|
||||
resetStats();
|
||||
return NextResponse.json({ success: true, stats: getBackgroundDegradationConfig().stats });
|
||||
}
|
||||
return NextResponse.json({ error: "Unknown action" }, { status: 400 });
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/background-degradation POST:", error);
|
||||
return NextResponse.json({ error: "Failed to execute action" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import {
|
||||
getAllAliases,
|
||||
getCustomAliases,
|
||||
getBuiltInAliases,
|
||||
setCustomAliases,
|
||||
addCustomAlias,
|
||||
removeCustomAlias,
|
||||
} from "@omniroute/open-sse/services/modelDeprecation.ts";
|
||||
import { getSettings, updateSettings } from "@/lib/db/settings";
|
||||
|
||||
/**
|
||||
* GET /api/settings/model-aliases
|
||||
* Returns the full alias map, separated into built-in and custom.
|
||||
*/
|
||||
export async function GET() {
|
||||
try {
|
||||
return NextResponse.json({
|
||||
builtIn: getBuiltInAliases(),
|
||||
custom: getCustomAliases(),
|
||||
all: getAllAliases(),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/model-aliases GET:", error);
|
||||
return NextResponse.json({ error: "Failed to get model aliases" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* PUT /api/settings/model-aliases
|
||||
* Update the custom aliases map.
|
||||
* Body: { aliases: { "old-model": "new-model", ... } }
|
||||
*/
|
||||
export async function PUT(request) {
|
||||
try {
|
||||
const { aliases } = await request.json();
|
||||
if (!aliases || typeof aliases !== "object") {
|
||||
return NextResponse.json({ error: "Missing or invalid 'aliases' object" }, { status: 400 });
|
||||
}
|
||||
setCustomAliases(aliases);
|
||||
await updateSettings({ modelAliases: JSON.stringify(aliases) });
|
||||
return NextResponse.json({ success: true, custom: getCustomAliases() });
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/model-aliases PUT:", error);
|
||||
return NextResponse.json({ error: "Failed to update model aliases" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/settings/model-aliases
|
||||
* Add a single custom alias.
|
||||
* Body: { from: "old-model", to: "new-model" }
|
||||
*/
|
||||
export async function POST(request) {
|
||||
try {
|
||||
const { from, to } = await request.json();
|
||||
if (!from || !to) {
|
||||
return NextResponse.json({ error: "Missing 'from' or 'to'" }, { status: 400 });
|
||||
}
|
||||
addCustomAlias(from, to);
|
||||
await updateSettings({ modelAliases: JSON.stringify(getCustomAliases()) });
|
||||
return NextResponse.json({ success: true, custom: getCustomAliases() });
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/model-aliases POST:", error);
|
||||
return NextResponse.json({ error: "Failed to add alias" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* DELETE /api/settings/model-aliases
|
||||
* Remove a custom alias.
|
||||
* Body: { from: "old-model" }
|
||||
*/
|
||||
export async function DELETE(request) {
|
||||
try {
|
||||
const { from } = await request.json();
|
||||
if (!from) {
|
||||
return NextResponse.json({ error: "Missing 'from'" }, { status: 400 });
|
||||
}
|
||||
const removed = removeCustomAlias(from);
|
||||
if (!removed) {
|
||||
return NextResponse.json({ error: "Alias not found" }, { status: 404 });
|
||||
}
|
||||
await updateSettings({ modelAliases: JSON.stringify(getCustomAliases()) });
|
||||
return NextResponse.json({ success: true, custom: getCustomAliases() });
|
||||
} catch (error) {
|
||||
console.error("[API ERROR] /api/settings/model-aliases DELETE:", error);
|
||||
return NextResponse.json({ error: "Failed to remove alias" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
deleteProxyForLevel,
|
||||
resolveProxyForConnection,
|
||||
} from "../../../../lib/localDb";
|
||||
import { clearDispatcherCache } from "@omniroute/open-sse/utils/proxyDispatcher";
|
||||
|
||||
const BASE_SUPPORTED_PROXY_TYPES = new Set(["http", "https"]);
|
||||
|
||||
@@ -129,6 +130,7 @@ export async function PUT(request) {
|
||||
const body = await request.json();
|
||||
const normalizedBody = normalizeProxyPayload(body);
|
||||
const updated = await setProxyConfig(normalizedBody);
|
||||
clearDispatcherCache();
|
||||
return Response.json(updated);
|
||||
} catch (error) {
|
||||
const status = Number(error?.status) || 500;
|
||||
@@ -155,6 +157,7 @@ export async function DELETE(request) {
|
||||
}
|
||||
|
||||
const updated = await deleteProxyForLevel(level, id);
|
||||
clearDispatcherCache();
|
||||
return Response.json(updated);
|
||||
} catch (error) {
|
||||
return Response.json(
|
||||
|
||||
@@ -114,11 +114,15 @@ async function syncAndVerify(machineId: string, createdKey: any, existingKeys: a
|
||||
);
|
||||
}
|
||||
|
||||
// Build the cloud URL for the frontend to use
|
||||
const cloudUrl = CLOUD_URL ? `${CLOUD_URL}/${machineId}` : null;
|
||||
|
||||
// Step 2: Verify connection by pinging the cloud (with retry)
|
||||
const apiKey = createdKey || existingKeys[0]?.key;
|
||||
if (!apiKey) {
|
||||
return NextResponse.json({
|
||||
...syncResult,
|
||||
cloudUrl,
|
||||
verified: false,
|
||||
verifyError: "No API key available",
|
||||
});
|
||||
@@ -146,6 +150,7 @@ async function syncAndVerify(machineId: string, createdKey: any, existingKeys: a
|
||||
if (pingResponse.ok) {
|
||||
return NextResponse.json({
|
||||
...syncResult,
|
||||
cloudUrl,
|
||||
verified: true,
|
||||
});
|
||||
}
|
||||
@@ -163,6 +168,7 @@ async function syncAndVerify(machineId: string, createdKey: any, existingKeys: a
|
||||
// Sync succeeded but verify failed — still return success with warning
|
||||
return NextResponse.json({
|
||||
...syncResult,
|
||||
cloudUrl,
|
||||
verified: false,
|
||||
verifyError: lastVerifyError || "Verification failed after retries",
|
||||
});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { CORS_ORIGIN } from "@/shared/utils/cors";
|
||||
import { handleAudioSpeech } from "@omniroute/open-sse/handlers/audioSpeech.ts";
|
||||
import { getProviderCredentials, extractApiKey, isValidApiKey } from "@/sse/services/auth";
|
||||
import { parseSpeechModel } from "@omniroute/open-sse/config/audioRegistry.ts";
|
||||
import { parseSpeechModel, getSpeechProvider } from "@omniroute/open-sse/config/audioRegistry.ts";
|
||||
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
|
||||
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import { enforceApiKeyPolicy } from "@/shared/utils/apiKeyPolicy";
|
||||
@@ -54,9 +54,16 @@ export async function POST(request) {
|
||||
);
|
||||
}
|
||||
|
||||
const credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${provider}`);
|
||||
// Check provider config for auth bypass
|
||||
const providerConfig = getSpeechProvider(provider);
|
||||
|
||||
// Get credentials — skip for local providers (authType: "none")
|
||||
let credentials = null;
|
||||
if (providerConfig && providerConfig.authType !== "none") {
|
||||
credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${provider}`);
|
||||
}
|
||||
}
|
||||
|
||||
return handleAudioSpeech({ body, credentials });
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { CORS_ORIGIN } from "@/shared/utils/cors";
|
||||
import { handleAudioTranscription } from "@omniroute/open-sse/handlers/audioTranscription.ts";
|
||||
import { getProviderCredentials, extractApiKey, isValidApiKey } from "@/sse/services/auth";
|
||||
import { parseTranscriptionModel } from "@omniroute/open-sse/config/audioRegistry.ts";
|
||||
import { parseTranscriptionModel, getTranscriptionProvider } from "@omniroute/open-sse/config/audioRegistry.ts";
|
||||
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
|
||||
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import { enforceApiKeyPolicy } from "@/shared/utils/apiKeyPolicy";
|
||||
@@ -56,9 +56,16 @@ export async function POST(request) {
|
||||
);
|
||||
}
|
||||
|
||||
const credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${provider}`);
|
||||
// Check provider config for auth bypass
|
||||
const providerConfig = getTranscriptionProvider(provider);
|
||||
|
||||
// Get credentials — skip for local providers (authType: "none")
|
||||
let credentials = null;
|
||||
if (providerConfig && providerConfig.authType !== "none") {
|
||||
credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${provider}`);
|
||||
}
|
||||
}
|
||||
|
||||
return handleAudioTranscription({ formData, credentials });
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { CORS_ORIGIN } from "@/shared/utils/cors";
|
||||
import { handleImageGeneration } from "@omniroute/open-sse/handlers/imageGeneration.ts";
|
||||
import { getProviderCredentials, extractApiKey, isValidApiKey } from "@/sse/services/auth";
|
||||
import { parseImageModel, getAllImageModels } from "@omniroute/open-sse/config/imageRegistry.ts";
|
||||
import { parseImageModel, getAllImageModels, getImageProvider } from "@omniroute/open-sse/config/imageRegistry.ts";
|
||||
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
|
||||
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import * as log from "@/sse/utils/logger";
|
||||
@@ -89,10 +89,16 @@ export async function POST(request) {
|
||||
);
|
||||
}
|
||||
|
||||
// Get credentials for the image provider
|
||||
const credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for image provider: ${provider}`);
|
||||
// Check provider config for auth bypass
|
||||
const providerConfig = getImageProvider(provider);
|
||||
|
||||
// Get credentials — skip for local providers (authType: "none")
|
||||
let credentials = null;
|
||||
if (providerConfig && providerConfig.authType !== "none") {
|
||||
credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for image provider: ${provider}`);
|
||||
}
|
||||
}
|
||||
|
||||
const result = await handleImageGeneration({ body, credentials, log });
|
||||
|
||||
@@ -14,6 +14,8 @@ import { getAllImageModels } from "@omniroute/open-sse/config/imageRegistry.ts";
|
||||
import { getAllRerankModels } from "@omniroute/open-sse/config/rerankRegistry.ts";
|
||||
import { getAllAudioModels } from "@omniroute/open-sse/config/audioRegistry.ts";
|
||||
import { getAllModerationModels } from "@omniroute/open-sse/config/moderationRegistry.ts";
|
||||
import { getAllVideoModels, getVideoProvider } from "@omniroute/open-sse/config/videoRegistry.ts";
|
||||
import { getAllMusicModels, getMusicProvider } from "@omniroute/open-sse/config/musicRegistry.ts";
|
||||
|
||||
const FALLBACK_ALIAS_TO_PROVIDER = {
|
||||
ag: "antigravity",
|
||||
@@ -310,6 +312,32 @@ export async function GET(request: Request) {
|
||||
});
|
||||
}
|
||||
|
||||
// Add video models (local providers always listed, cloud filtered by active)
|
||||
for (const videoModel of getAllVideoModels()) {
|
||||
const vConfig = getVideoProvider(videoModel.provider);
|
||||
if (vConfig?.authType !== "none" && !isProviderActive(videoModel.provider)) continue;
|
||||
models.push({
|
||||
id: videoModel.id,
|
||||
object: "model",
|
||||
created: timestamp,
|
||||
owned_by: videoModel.provider,
|
||||
type: "video",
|
||||
});
|
||||
}
|
||||
|
||||
// Add music models (local providers always listed, cloud filtered by active)
|
||||
for (const musicModel of getAllMusicModels()) {
|
||||
const mConfig = getMusicProvider(musicModel.provider);
|
||||
if (mConfig?.authType !== "none" && !isProviderActive(musicModel.provider)) continue;
|
||||
models.push({
|
||||
id: musicModel.id,
|
||||
object: "model",
|
||||
created: timestamp,
|
||||
owned_by: musicModel.provider,
|
||||
type: "music",
|
||||
});
|
||||
}
|
||||
|
||||
// Add custom models (user-defined)
|
||||
try {
|
||||
const customModelsMap: Record<string, any[]> = await getAllCustomModels();
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
import { CORS_ORIGIN } from "@/shared/utils/cors";
|
||||
import { handleMusicGeneration } from "@omniroute/open-sse/handlers/musicGeneration.ts";
|
||||
import { getProviderCredentials, extractApiKey, isValidApiKey } from "@/sse/services/auth";
|
||||
import { parseMusicModel, getAllMusicModels, getMusicProvider } from "@omniroute/open-sse/config/musicRegistry.ts";
|
||||
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
|
||||
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import * as log from "@/sse/utils/logger";
|
||||
import { toJsonErrorPayload } from "@/shared/utils/upstreamError";
|
||||
import { enforceApiKeyPolicy } from "@/shared/utils/apiKeyPolicy";
|
||||
|
||||
/**
|
||||
* Handle CORS preflight
|
||||
*/
|
||||
export async function OPTIONS() {
|
||||
return new Response(null, {
|
||||
headers: {
|
||||
"Access-Control-Allow-Origin": CORS_ORIGIN,
|
||||
"Access-Control-Allow-Methods": "GET, POST, OPTIONS",
|
||||
"Access-Control-Allow-Headers": "*",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /v1/music/generations — list available music models
|
||||
*/
|
||||
export async function GET() {
|
||||
const models = getAllMusicModels();
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
object: "list",
|
||||
data: models.map((m) => ({
|
||||
id: m.id,
|
||||
object: "model",
|
||||
created: Math.floor(Date.now() / 1000),
|
||||
owned_by: m.provider,
|
||||
type: "music",
|
||||
})),
|
||||
}),
|
||||
{
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /v1/music/generations — generate music
|
||||
*/
|
||||
export async function POST(request) {
|
||||
let body;
|
||||
try {
|
||||
body = await request.json();
|
||||
} catch {
|
||||
log.warn("MUSIC", "Invalid JSON body");
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Invalid JSON body");
|
||||
}
|
||||
|
||||
// Optional API key validation
|
||||
if (process.env.REQUIRE_API_KEY === "true") {
|
||||
const apiKey = extractApiKey(request);
|
||||
if (!apiKey) {
|
||||
return errorResponse(HTTP_STATUS.UNAUTHORIZED, "Missing API key");
|
||||
}
|
||||
const valid = await isValidApiKey(apiKey);
|
||||
if (!valid) {
|
||||
return errorResponse(HTTP_STATUS.UNAUTHORIZED, "Invalid API key");
|
||||
}
|
||||
}
|
||||
|
||||
if (!body.model) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Missing model");
|
||||
}
|
||||
|
||||
if (typeof body.prompt !== "string" || body.prompt.trim().length === 0) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Invalid prompt: expected a non-empty string");
|
||||
}
|
||||
|
||||
// Enforce API key policies (model restrictions + budget limits)
|
||||
const policy = await enforceApiKeyPolicy(request, body.model);
|
||||
if (policy.rejection) return policy.rejection;
|
||||
|
||||
// Parse model to get provider
|
||||
const { provider } = parseMusicModel(body.model);
|
||||
if (!provider) {
|
||||
return errorResponse(
|
||||
HTTP_STATUS.BAD_REQUEST,
|
||||
`Invalid music model: ${body.model}. Use format: provider/model`
|
||||
);
|
||||
}
|
||||
|
||||
// Check provider config for auth bypass
|
||||
const providerConfig = getMusicProvider(provider);
|
||||
|
||||
// Get credentials — skip for local providers (authType: "none")
|
||||
let credentials = null;
|
||||
if (providerConfig && providerConfig.authType !== "none") {
|
||||
credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for music provider: ${provider}`);
|
||||
}
|
||||
}
|
||||
|
||||
const result = await handleMusicGeneration({ body, credentials, log });
|
||||
|
||||
if (result.success) {
|
||||
return new Response(JSON.stringify((result as any).data), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
const errorPayload = toJsonErrorPayload((result as any).error, "Music generation provider error");
|
||||
return new Response(JSON.stringify(errorPayload), {
|
||||
status: (result as any).status,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
import { CORS_ORIGIN } from "@/shared/utils/cors";
|
||||
import { handleVideoGeneration } from "@omniroute/open-sse/handlers/videoGeneration.ts";
|
||||
import { getProviderCredentials, extractApiKey, isValidApiKey } from "@/sse/services/auth";
|
||||
import { parseVideoModel, getAllVideoModels, getVideoProvider } from "@omniroute/open-sse/config/videoRegistry.ts";
|
||||
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
|
||||
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import * as log from "@/sse/utils/logger";
|
||||
import { toJsonErrorPayload } from "@/shared/utils/upstreamError";
|
||||
import { enforceApiKeyPolicy } from "@/shared/utils/apiKeyPolicy";
|
||||
|
||||
/**
|
||||
* Handle CORS preflight
|
||||
*/
|
||||
export async function OPTIONS() {
|
||||
return new Response(null, {
|
||||
headers: {
|
||||
"Access-Control-Allow-Origin": CORS_ORIGIN,
|
||||
"Access-Control-Allow-Methods": "GET, POST, OPTIONS",
|
||||
"Access-Control-Allow-Headers": "*",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /v1/videos/generations — list available video models
|
||||
*/
|
||||
export async function GET() {
|
||||
const models = getAllVideoModels();
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
object: "list",
|
||||
data: models.map((m) => ({
|
||||
id: m.id,
|
||||
object: "model",
|
||||
created: Math.floor(Date.now() / 1000),
|
||||
owned_by: m.provider,
|
||||
type: "video",
|
||||
})),
|
||||
}),
|
||||
{
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /v1/videos/generations — generate videos
|
||||
*/
|
||||
export async function POST(request) {
|
||||
let body;
|
||||
try {
|
||||
body = await request.json();
|
||||
} catch {
|
||||
log.warn("VIDEO", "Invalid JSON body");
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Invalid JSON body");
|
||||
}
|
||||
|
||||
// Optional API key validation
|
||||
if (process.env.REQUIRE_API_KEY === "true") {
|
||||
const apiKey = extractApiKey(request);
|
||||
if (!apiKey) {
|
||||
return errorResponse(HTTP_STATUS.UNAUTHORIZED, "Missing API key");
|
||||
}
|
||||
const valid = await isValidApiKey(apiKey);
|
||||
if (!valid) {
|
||||
return errorResponse(HTTP_STATUS.UNAUTHORIZED, "Invalid API key");
|
||||
}
|
||||
}
|
||||
|
||||
if (!body.model) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Missing model");
|
||||
}
|
||||
|
||||
if (typeof body.prompt !== "string" || body.prompt.trim().length === 0) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, "Invalid prompt: expected a non-empty string");
|
||||
}
|
||||
|
||||
// Enforce API key policies (model restrictions + budget limits)
|
||||
const policy = await enforceApiKeyPolicy(request, body.model);
|
||||
if (policy.rejection) return policy.rejection;
|
||||
|
||||
// Parse model to get provider
|
||||
const { provider } = parseVideoModel(body.model);
|
||||
if (!provider) {
|
||||
return errorResponse(
|
||||
HTTP_STATUS.BAD_REQUEST,
|
||||
`Invalid video model: ${body.model}. Use format: provider/model`
|
||||
);
|
||||
}
|
||||
|
||||
// Check provider config for auth bypass
|
||||
const providerConfig = getVideoProvider(provider);
|
||||
|
||||
// Get credentials — skip for local providers (authType: "none")
|
||||
let credentials = null;
|
||||
if (providerConfig && providerConfig.authType !== "none") {
|
||||
credentials = await getProviderCredentials(provider);
|
||||
if (!credentials) {
|
||||
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for video provider: ${provider}`);
|
||||
}
|
||||
}
|
||||
|
||||
const result = await handleVideoGeneration({ body, credentials, log });
|
||||
|
||||
if (result.success) {
|
||||
return new Response(JSON.stringify((result as any).data), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
const errorPayload = toJsonErrorPayload((result as any).error, "Video generation provider error");
|
||||
return new Response(JSON.stringify(errorPayload), {
|
||||
status: (result as any).status,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
+9
-2
@@ -1,6 +1,12 @@
|
||||
@import url("https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:wght,FILL@100..700,0..1&display=swap");
|
||||
@import "tailwindcss";
|
||||
|
||||
/* Tailwind v4 auto-detection cannot scan directories with parentheses
|
||||
(e.g. Next.js route groups like "(dashboard)"). Explicit @source
|
||||
directives ensure all utility classes in route groups are included. */
|
||||
@source "../app/(dashboard)";
|
||||
@source "../../open-sse";
|
||||
|
||||
@custom-variant dark (&:where(.dark, .dark *));
|
||||
|
||||
/* OpenClaw × ClawHub Color Palette */
|
||||
@@ -135,9 +141,10 @@ body {
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
}
|
||||
|
||||
/* Selection */
|
||||
/* Selection — with fallback for browsers that don't support color-mix() */
|
||||
::selection {
|
||||
background-color: rgba(229, 77, 94, 0.2);
|
||||
background-color: rgba(229, 77, 94, 0.22);
|
||||
background-color: color-mix(in srgb, var(--color-primary) 22%, transparent);
|
||||
color: var(--color-primary);
|
||||
}
|
||||
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@
|
||||
*/
|
||||
|
||||
/**
|
||||
* @typedef {'openai'|'claude'|'gemini'|'codex'|'qwen'|'deepseek'|'cohere'|'groq'|'mistral'|'openrouter'} ProviderId
|
||||
* @typedef {'openai'|'claude'|'gemini'|'codex'|'qwen'|'deepseek'|'cohere'|'groq'|'blackbox'|'mistral'|'openrouter'} ProviderId
|
||||
*/
|
||||
|
||||
/**
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user