Compare commits

..

9 Commits

Author SHA1 Message Date
diegosouzapw 48754fd999 release: v2.8.7 — Bottleneck 429 drop (PR #495), custom embedding provider fix (#496)
Build Electron Desktop App / Validate version (push) Failing after 33s
Build Electron Desktop App / Build Electron (macos-arm64) (push) Has been skipped
Build Electron Desktop App / Build Electron (linux) (push) Has been skipped
Build Electron Desktop App / Build Electron (macos-intel) (push) Has been skipped
Build Electron Desktop App / Build Electron (windows) (push) Has been skipped
Build Electron Desktop App / Create Release (push) Has been skipped
2026-03-20 12:57:08 -03:00
Diego Rodrigues de Sa e Souza c496ebdef9 Merge pull request #495 from xandr0s/fix/429-drop-bottleneck-queue
fix: drop Bottleneck queue on 429 instead of infinite wait
2026-03-20 12:53:31 -03:00
Oleg Saprykin c009c40606 refactor: use .finally() to always delete limiter from Map
Address bot review feedback: use .finally() instead of .then()/.catch()
so limiters.delete() runs regardless of whether stop() succeeds or
throws (e.g. already stopped by concurrent 429).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 18:31:36 +03:00
Oleg Saprykin b29456c8e5 fix: catch stop() already called on concurrent 429s
Multiple concurrent requests can receive 429 simultaneously, causing
stop() to be called on an already-stopped limiter. Add .catch() to
prevent unhandled rejection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 18:27:46 +03:00
diegosouzapw 38266bf2ff release: v2.8.6 — MiniMax role fix (PR #494), KIRO MITM card (#487), triage 8 issues
Build Electron Desktop App / Validate version (push) Failing after 29s
Build Electron Desktop App / Build Electron (macos-arm64) (push) Has been skipped
Build Electron Desktop App / Build Electron (linux) (push) Has been skipped
Build Electron Desktop App / Build Electron (macos-intel) (push) Has been skipped
Build Electron Desktop App / Build Electron (windows) (push) Has been skipped
Build Electron Desktop App / Create Release (push) Has been skipped
2026-03-20 12:26:27 -03:00
Diego Rodrigues de Sa e Souza c2e51f8948 Merge pull request #494 from zhangqiang8vip/fix/developer-role-param-error
fix: resolve 422 "role param error" when forwarding OpenAI Responses API to MiniMax (developer → system)
2026-03-20 12:21:57 -03:00
Oleg Saprykin 64f040bddd fix: drop Bottleneck queue on 429 instead of waiting for reservoir refresh
When a provider returns 429 (rate limit exceeded), the rate limit manager
was setting reservoir=0 and waiting for reservoirRefreshInterval before
releasing queued requests. For providers with long rate limit windows
(e.g. Codex with hours-long resets), this caused all queued requests to
hang indefinitely — they never timed out or returned an error.

This prevented upstream callers (e.g. LiteLLM) from triggering fallback
to alternative providers, effectively making the entire model unavailable
until the rate limit window expired.

Fix: on 429, call limiter.stop({ dropWaitingJobs: true }) to immediately
fail all queued requests, then delete the limiter from the Map so
getLimiter() creates a fresh instance for subsequent requests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 18:07:56 +03:00
zhang-qiang dfbb9d5fff docs: add ZWS_README_V2 — developer role fix documentation
Made-with: Cursor
2026-03-20 21:47:02 +08:00
zhang-qiang a7fe369ea0 fix: resolve role param error for Responses API + MiniMax (developer→system)
- Add preserveDeveloperRole option and model compat override

- Normalize developer→system in roleNormalizer when not preserving

- Translator runs normalizeRoles for Responses API with option

- UI: ModelCompatPopover with do not preserve developer toggle

- Add ZWS_README_V2 documenting cause and fix

Made-with: Cursor
2026-03-20 21:06:10 +08:00
18 changed files with 745 additions and 122 deletions
+50
View File
@@ -4,6 +4,56 @@
---
## [2.8.7] — 2026-03-20
> Sprint: Merge PR #495 (Bottleneck 429 drop), fix #496 (custom embedding providers), triage features.
### Bug Fixes
- **Bottleneck 429 infinite wait** (PR #495 by @xandr0s): On 429, `limiter.stop({ dropWaitingJobs: true })` immediately fails all queued requests so upstream callers can trigger fallback. Limiter is deleted from Map so next request creates a fresh instance.
- **Custom embedding models unresolvable** (#496): `POST /v1/embeddings` now resolves custom embedding models from ALL provider_nodes (not just localhost). Enables models like `google/gemini-embedding-001` added via dashboard.
### Issues Responded
- **#452** — Per-API-key request-count limits (acknowledged, on roadmap)
- **#464** — Auto-issue API keys with provider/account limits (needs more detail)
- **#488** — Auto-update model lists (acknowledged, on roadmap)
- **#496** — Custom embedding provider resolution (fixed)
---
## [2.8.6] — 2026-03-20
> Sprint: Merge PR #494 (MiniMax role fix), fix KIRO MITM dashboard, triage 8 issues.
### Features
- **MiniMax developer→system role fix** (PR #494 by @zhangqiang8vip): Per-model `preserveDeveloperRole` toggle. Adds "Compatibility" UI in providers page. Fixes 422 "role param error" for MiniMax and similar gateways.
- **roleNormalizer**: `normalizeDeveloperRole()` now accepts `preserveDeveloperRole` parameter with tri-state behavior (undefined=keep, true=keep, false=convert).
- **DB**: New `getModelPreserveOpenAIDeveloperRole()` and `mergeModelCompatOverride()` in `models.ts`.
### Bug Fixes
- **KIRO MITM dashboard** (#481/#487): `CLIToolsPageClient` now routes any `configType: "mitm"` tool to `AntigravityToolCard` (MITM Start/Stop controls). Previously only Antigravity was hardcoded.
- **AntigravityToolCard generic**: Uses `tool.image`, `tool.description`, `tool.id` instead of hardcoded Antigravity values. Guards against missing `defaultModels`.
### Cleanup
- Removed `ZWS_README_V2.md` (development-only docs from PR #494).
### Issues Triaged (8)
- **#487** — Closed (KIRO MITM fixed in this release)
- **#486** — needs-info (Windows REG.exe PATH issue)
- **#489** — needs-info (Antigravity projectId missing, OAuth reconnect needed)
- **#492** — needs-info (missing app/server.js on mise-managed Node)
- **#490** — Acknowledged (streaming + context cache blocking, fix planned)
- **#491** — Acknowledged (Codex auth state inconsistency)
- **#493** — Acknowledged (Modal provider model name prefix, workaround provided)
- **#488** — Feature request backlog (auto-update model lists)
---
## [2.8.5] — 2026-03-19
> Sprint: Fix zombie SSE streams, context cache first-turn, KIRO MITM, and triage 5 external issues.
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.1.0
info:
title: OmniRoute API
version: 2.8.5
version: 2.8.7
description: |
OmniRoute is a local-first AI API proxy router. It provides an OpenAI-compatible
endpoint that routes requests to multiple AI providers with load balancing,
+6 -2
View File
@@ -23,7 +23,7 @@ import {
appendRequestLog,
saveCallLog,
} from "@/lib/usageDb";
import { getModelNormalizeToolCallId } from "@/lib/db/models";
import { getModelNormalizeToolCallId, getModelPreserveOpenAIDeveloperRole } from "@/lib/localDb";
import { getExecutor } from "../executors/index.ts";
import { translateNonStreamingResponse } from "./responseTranslator.ts";
import { extractUsageFromResponse } from "./usageExtractor.ts";
@@ -318,6 +318,10 @@ export async function handleChatCore({
}
const normalizeToolCallId = getModelNormalizeToolCallId(provider || "", model || "");
const preserveDeveloperRole = getModelPreserveOpenAIDeveloperRole(
provider || "",
model || ""
);
translatedBody = translateRequest(
sourceFormat,
targetFormat,
@@ -327,7 +331,7 @@ export async function handleChatCore({
credentials,
provider,
reqLogger,
{ normalizeToolCallId }
{ normalizeToolCallId, preserveDeveloperRole }
);
}
} catch (error) {
+10 -5
View File
@@ -339,14 +339,19 @@ export function updateFromHeaders(provider, connectionId, headers, status, model
// Handle 429 — rate limited
if (status === 429) {
const retryAfterMs = parseResetTime(retryAfterStr) || 60000; // Default 60s
const counts = limiter.counts();
const limiterKey = `${provider}:${connectionId}`;
console.log(
`🚫 [RATE-LIMIT] ${provider}:${connectionId.slice(0, 8)} — 429 received, pausing for ${Math.ceil(retryAfterMs / 1000)}s`
`🚫 [RATE-LIMIT] ${provider}:${connectionId.slice(0, 8)} — 429 received, pausing for ${Math.ceil(retryAfterMs / 1000)}s, dropping ${counts.QUEUED} queued request(s)`
);
limiter.updateSettings({
reservoir: 0,
reservoirRefreshAmount: limit || 60,
reservoirRefreshInterval: retryAfterMs,
// Stop the limiter and drop all waiting jobs so they fail immediately
// instead of hanging in the queue until reservoir refreshes (which can
// be hours for providers like Codex with long rate limit windows).
// This lets upstream callers (e.g. LiteLLM) trigger fallback to other providers.
// After stop, delete from Map so getLimiter() creates a fresh instance.
limiter.stop({ dropWaitingJobs: true }).finally(() => {
limiters.delete(limiterKey);
});
return;
}
+14 -19
View File
@@ -76,26 +76,27 @@ function supportsSystemRole(provider: string, model: string): boolean {
}
/**
* Normalize the `developer` role to `system` for non-OpenAI providers.
* OpenAI introduced `developer` as a replacement for `system` in newer models,
* but most other providers still expect `system`.
* Normalize the `developer` role to `system` when the upstream does not support it.
* OpenAI Responses API sends `developer`; MiniMax and most OpenAI-compatible gateways
* only accept system/user/assistant/tool and return "role param error" otherwise.
*
* @param messages - Array of messages
* @param targetFormat - The target format (e.g., "openai", "claude", "gemini")
* @returns Modified messages array
* @param preserveDeveloperRole - For targetFormat openai: undefined/true = keep developer (legacy default); false = map to system (MiniMax etc.)
*/
export function normalizeDeveloperRole(
messages: NormalizedMessage[] | unknown,
targetFormat: string
targetFormat: string,
preserveDeveloperRole?: boolean
): NormalizedMessage[] | unknown {
if (!Array.isArray(messages)) return messages;
// For OpenAI format, keep developer role as-is (it's valid)
// For all other formats, convert developer → system
if (targetFormat === "openai") return messages;
if (targetFormat === "openai" && preserveDeveloperRole !== false) return messages;
return messages.map((msg: NormalizedMessage) => {
if (msg.role === "developer") {
if (!msg || typeof msg !== "object") return msg;
const role = typeof msg.role === "string" ? msg.role : "";
if (role.toLowerCase() === "developer") {
return { ...msg, role: "system" };
}
return msg;
@@ -170,24 +171,18 @@ export function normalizeSystemRole(
* Full role normalization pipeline.
* Call this before sending the request to the provider.
*
* @param messages - Array of messages
* @param provider - Provider name/id
* @param model - Model name
* @param targetFormat - Target API format
* @returns Normalized messages array
* @param preserveDeveloperRole - See {@link normalizeDeveloperRole}
*/
export function normalizeRoles(
messages: NormalizedMessage[] | unknown,
provider: string,
model: string,
targetFormat: string
targetFormat: string,
preserveDeveloperRole?: boolean
): NormalizedMessage[] | unknown {
if (!Array.isArray(messages)) return messages;
// Step 1: Normalize developer → system (for non-OpenAI formats)
let result = normalizeDeveloperRole(messages, targetFormat);
// Step 2: Normalize system → user (for providers that don't support system role)
let result = normalizeDeveloperRole(messages, targetFormat, preserveDeveloperRole);
result = normalizeSystemRole(result, provider, model);
return result;
+27 -3
View File
@@ -67,6 +67,7 @@ function normalizeOpenAIResponsesRequest(body) {
}
/** @param options.normalizeToolCallId - When true, use 9-char tool call ids (e.g. Mistral); when false, leave ids as-is */
/** @param options.preserveDeveloperRole - undefined/true: keep developer for OpenAI format (default); false: map to system */
// Translate request: source -> openai -> target
export function translateRequest(
sourceFormat,
@@ -77,10 +78,11 @@ export function translateRequest(
credentials = null,
provider = null,
reqLogger = null,
options?: { normalizeToolCallId?: boolean }
options?: { normalizeToolCallId?: boolean; preserveDeveloperRole?: boolean }
) {
let result = body;
const use9CharId = options?.normalizeToolCallId === true;
const preserveDeveloperRole = options?.preserveDeveloperRole;
// Phase 2: Apply thinking budget control before normalization
result = applyThinkingBudget(result);
@@ -94,9 +96,15 @@ export function translateRequest(
// Fix missing tool responses (insert empty tool_result if needed)
fixMissingToolResponses(result);
// Normalize roles: developer→system for non-OpenAI, system→user for incompatible models
// Normalize roles: developer→system unless preserved, system→user for incompatible models
if (result.messages && Array.isArray(result.messages)) {
result.messages = normalizeRoles(result.messages, provider || "", model || "", targetFormat);
result.messages = normalizeRoles(
result.messages,
provider || "",
model || "",
targetFormat,
preserveDeveloperRole
);
}
// If same format, skip translation steps
@@ -143,6 +151,22 @@ export function translateRequest(
result = normalizeOpenAIResponsesRequest(result);
}
// After OPENAI_RESPONSES → OPENAI, messages are built from input; first normalizeRoles was a no-op.
// Run role pipeline again so developer→system respects preserveDeveloperRole (no hardcoding in translator).
if (
sourceFormat === FORMATS.OPENAI_RESPONSES &&
result.messages &&
Array.isArray(result.messages)
) {
result.messages = normalizeRoles(
result.messages,
provider || "",
model || "",
targetFormat,
preserveDeveloperRole
);
}
// Ensure unique tool_call ids on final payload (translators may have introduced duplicates)
ensureToolCallIds(result, { use9CharId });
fixMissingToolResponses(result);
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "omniroute",
"version": "2.8.5",
"version": "2.8.7",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "omniroute",
"version": "2.8.5",
"version": "2.8.7",
"hasInstallScript": true,
"license": "MIT",
"workspaces": [
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "omniroute",
"version": "2.8.5",
"version": "2.8.7",
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
"type": "module",
"bin": {
@@ -267,6 +267,18 @@ export default function CLIToolsPageClient({ machineId }) {
/>
);
default:
// #487: Any tool with configType "mitm" should use the MITM card (Start/Stop controls)
if (tool.configType === "mitm") {
return (
<AntigravityToolCard
key={toolId}
{...commonProps}
activeProviders={getActiveProviders()}
hasActiveProviders={hasActiveProviders}
cloudEnabled={cloudEnabled}
/>
);
}
return (
<DefaultToolCard
key={toolId}
@@ -41,7 +41,7 @@ export default function AntigravityToolCard({
const loadSavedMappings = async () => {
try {
const res = await fetch("/api/cli-tools/antigravity-mitm/alias?tool=antigravity");
const res = await fetch(`/api/cli-tools/antigravity-mitm/alias?tool=${tool.id}`);
if (res.ok) {
const data = await res.json();
const aliases = data.aliases || {};
@@ -187,7 +187,7 @@ export default function AntigravityToolCard({
const res = await fetch("/api/cli-tools/antigravity-mitm/alias", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ tool: "antigravity", mappings: modelMappings }),
body: JSON.stringify({ tool: tool.id, mappings: modelMappings }),
});
if (!res.ok) {
@@ -211,7 +211,7 @@ export default function AntigravityToolCard({
<div className="flex items-center gap-3">
<div className="size-8 flex items-center justify-center shrink-0">
<Image
src="/providers/antigravity.png"
src={tool.image || "/providers/antigravity.png"}
alt={tool.name}
width={32}
height={32}
@@ -235,7 +235,7 @@ export default function AntigravityToolCard({
</Badge>
)}
</div>
<p className="text-xs text-text-muted truncate">{t("toolDescriptions.antigravity")}</p>
<p className="text-xs text-text-muted truncate">{tool.description}</p>
</div>
</div>
<span
@@ -306,7 +306,7 @@ export default function AntigravityToolCard({
)}
</div>
{tool.defaultModels.map((model) => (
{(tool.defaultModels || []).map((model) => (
<div key={model.alias} className="flex items-center gap-2">
<span className="w-32 shrink-0 text-sm font-semibold text-text-main text-right">
{model.name}
@@ -43,6 +43,78 @@ function normalizeCodexLimitPolicy(policy: unknown): { use5h: boolean; useWeekly
};
}
function ModelCompatPopover({
t,
normalizeToolCallId,
preserveDeveloperRole,
showDeveloperToggle = true,
onNormalizeChange,
onPreserveChange,
disabled,
}: {
t: (key: string) => string;
normalizeToolCallId: boolean;
preserveDeveloperRole?: boolean;
showDeveloperToggle?: boolean;
onNormalizeChange: (v: boolean) => void;
onPreserveChange: (v: boolean) => void;
disabled?: boolean;
}) {
const [open, setOpen] = useState(false);
const ref = useRef<HTMLDivElement>(null);
useEffect(() => {
if (!open) return;
const onDocClick = (e: MouseEvent) => {
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
};
document.addEventListener("mousedown", onDocClick);
return () => document.removeEventListener("mousedown", onDocClick);
}, [open]);
return (
<div className="relative inline-block" ref={ref}>
<button
type="button"
onClick={() => setOpen((v) => !v)}
disabled={disabled}
className="inline-flex items-center gap-1 px-2 py-1 text-xs rounded-md border border-border bg-sidebar/50 hover:bg-sidebar text-text-muted hover:text-text-main disabled:opacity-50"
title={t("compatAdjustmentsTitle")}
>
<span className="material-symbols-outlined text-sm">tune</span>
{t("compatButtonLabel")}
</button>
{open && (
<div className="absolute left-0 top-full mt-1 z-50 min-w-[200px] p-3 rounded-lg border border-border bg-white dark:bg-zinc-900 shadow-xl ring-1 ring-black/5 dark:ring-white/10">
<p className="text-[10px] font-semibold uppercase tracking-wide text-text-muted mb-2">
{t("compatAdjustmentsTitle")}
</p>
<div className="flex flex-col gap-3">
<Toggle
size="sm"
label={t("compatToolIdShort")}
title={t("normalizeToolCallIdLabel")}
checked={normalizeToolCallId}
onChange={onNormalizeChange}
disabled={disabled}
/>
{showDeveloperToggle && (
<Toggle
size="sm"
label={t("compatDoNotPreserveDeveloper")}
title={t("preserveDeveloperRoleLabel")}
checked={preserveDeveloperRole === false}
onChange={(checked) => onPreserveChange(!checked)}
disabled={disabled}
/>
)}
</div>
</div>
)}
</div>
);
}
export default function ProviderDetailPage() {
const params = useParams();
const router = useRouter();
@@ -76,6 +148,15 @@ export default function ProviderDetailPage() {
error: "",
importedCount: 0,
});
const [modelMeta, setModelMeta] = useState<{
customModels: Record<string, unknown>[];
modelCompatOverrides: {
id: string;
normalizeToolCallId?: boolean;
preserveOpenAIDeveloperRole?: boolean;
}[];
}>({ customModels: [], modelCompatOverrides: [] });
const [compatSavingModelId, setCompatSavingModelId] = useState<string | null>(null);
const providerInfo = providerNode
? {
@@ -119,6 +200,23 @@ export default function ProviderDetailPage() {
}
}, []);
const fetchProviderModelMeta = useCallback(async () => {
if (isSearchProvider) return;
try {
const res = await fetch(`/api/provider-models?provider=${encodeURIComponent(providerId)}`, {
cache: "no-store",
});
if (!res.ok) return;
const data = await res.json();
setModelMeta({
customModels: data.models || [],
modelCompatOverrides: data.modelCompatOverrides || [],
});
} catch (e) {
console.error("fetchProviderModelMeta", e);
}
}, [providerId, isSearchProvider]);
const fetchConnections = useCallback(async () => {
try {
const [connectionsRes, nodesRes] = await Promise.all([
@@ -186,6 +284,11 @@ export default function ProviderDetailPage() {
.catch(() => {});
}, [fetchConnections, fetchAliases]);
useEffect(() => {
if (loading || isSearchProvider) return;
fetchProviderModelMeta();
}, [loading, isSearchProvider, fetchProviderModelMeta]);
// Auto-open Add Connection modal when no connections exist (better UX)
// Only fires once on initial load, not on HMR remounts or after user dismissal
useEffect(() => {
@@ -670,6 +773,79 @@ export default function ProviderDetailPage() {
const canImportModels = connections.some((conn) => conn.isActive !== false);
const effectiveModelNormalize = (modelId: string) => {
const c = modelMeta.customModels.find((m: { id?: string }) => m.id === modelId) as
| { normalizeToolCallId?: boolean }
| undefined;
if (c) return Boolean(c.normalizeToolCallId);
const o = modelMeta.modelCompatOverrides.find((e) => e.id === modelId);
return Boolean(o?.normalizeToolCallId);
};
const effectiveModelPreserveDeveloper = (modelId: string) => {
const c = modelMeta.customModels.find((m: { id?: string }) => m.id === modelId) as
| Record<string, unknown>
| undefined;
if (c && Object.prototype.hasOwnProperty.call(c, "preserveOpenAIDeveloperRole")) {
return Boolean(c.preserveOpenAIDeveloperRole);
}
const o = modelMeta.modelCompatOverrides.find((e) => e.id === modelId);
if (o && Object.prototype.hasOwnProperty.call(o, "preserveOpenAIDeveloperRole")) {
return Boolean(o.preserveOpenAIDeveloperRole);
}
return true;
};
const saveModelCompatFlags = async (
modelId: string,
patch: { normalizeToolCallId?: boolean; preserveOpenAIDeveloperRole?: boolean }
) => {
setCompatSavingModelId(modelId);
try {
const c = modelMeta.customModels.find((m: { id?: string }) => m.id === modelId) as Record<
string,
unknown
> | null;
let body: Record<string, unknown>;
if (c) {
body = {
provider: providerId,
modelId,
modelName: (c.name as string) || modelId,
source: (c.source as string) || "manual",
apiFormat: (c.apiFormat as string) || "chat-completions",
supportedEndpoints:
Array.isArray(c.supportedEndpoints) && (c.supportedEndpoints as unknown[]).length
? c.supportedEndpoints
: ["chat"],
normalizeToolCallId:
patch.normalizeToolCallId !== undefined
? patch.normalizeToolCallId
: Boolean(c.normalizeToolCallId),
preserveOpenAIDeveloperRole:
patch.preserveOpenAIDeveloperRole !== undefined
? patch.preserveOpenAIDeveloperRole
: Object.prototype.hasOwnProperty.call(c, "preserveOpenAIDeveloperRole")
? Boolean(c.preserveOpenAIDeveloperRole)
: true,
};
} else {
body = { provider: providerId, modelId, ...patch };
}
const res = await fetch("/api/provider-models", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
if (res.ok) await fetchProviderModelMeta();
else notify.error(t("failedSaveCustomModel"));
} catch {
notify.error(t("failedSaveCustomModel"));
} finally {
setCompatSavingModelId(null);
}
};
const renderModelsSection = () => {
if (isCompatible) {
return (
@@ -684,6 +860,12 @@ export default function ProviderDetailPage() {
connections={connections}
isAnthropic={isAnthropicCompatible}
onImportWithProgress={handleCompatibleImportWithProgress}
t={t}
effectiveModelNormalize={effectiveModelNormalize}
effectiveModelPreserveDeveloper={effectiveModelPreserveDeveloper}
saveModelCompatFlags={saveModelCompatFlags}
compatSavingModelId={compatSavingModelId}
onModelsChanged={fetchProviderModelMeta}
/>
);
}
@@ -711,6 +893,11 @@ export default function ProviderDetailPage() {
onCopy={copy}
onSetAlias={handleSetAlias}
onDeleteAlias={handleDeleteAlias}
t={t}
effectiveModelNormalize={effectiveModelNormalize}
effectiveModelPreserveDeveloper={effectiveModelPreserveDeveloper}
saveModelCompatFlags={saveModelCompatFlags}
compatSavingModelId={compatSavingModelId}
/>
</div>
);
@@ -759,8 +946,17 @@ export default function ProviderDetailPage() {
alias={existingAlias}
copied={copied}
onCopy={copy}
onSetAlias={(alias) => handleSetAlias(model.id, alias, providerStorageAlias)}
onDeleteAlias={() => handleDeleteAlias(existingAlias)}
t={t}
showDeveloperToggle
normalizeToolCallId={effectiveModelNormalize(model.id)}
preserveDeveloperRole={effectiveModelPreserveDeveloper(model.id)}
onNormalizeChange={(v) =>
saveModelCompatFlags(model.id, { normalizeToolCallId: v })
}
onPreserveChange={(v) =>
saveModelCompatFlags(model.id, { preserveOpenAIDeveloperRole: v })
}
compatDisabled={compatSavingModelId === model.id}
/>
);
})}
@@ -1078,6 +1274,7 @@ export default function ProviderDetailPage() {
providerAlias={providerDisplayAlias}
copied={copied}
onCopy={copy}
onModelsChanged={fetchProviderModelMeta}
/>
)}
</Card>
@@ -1282,23 +1479,48 @@ export default function ProviderDetailPage() {
);
}
function ModelRow({ model, fullModel, alias, copied, onCopy, onSetAlias, onDeleteAlias }: any) {
const t = useTranslations("providers");
function ModelRow({
model,
fullModel,
alias,
copied,
onCopy,
t,
showDeveloperToggle = true,
normalizeToolCallId,
preserveDeveloperRole,
onNormalizeChange,
onPreserveChange,
compatDisabled,
}: any) {
return (
<div className="flex items-center gap-2 px-3 py-2 rounded-lg border border-border hover:bg-sidebar/50">
<span className="material-symbols-outlined text-base text-text-muted">smart_toy</span>
<code className="text-xs text-text-muted font-mono bg-sidebar px-1.5 py-0.5 rounded">
{fullModel}
</code>
<button
onClick={() => onCopy(fullModel, `model-${model.id}`)}
className="p-0.5 hover:bg-sidebar rounded text-text-muted hover:text-primary"
title={t("copyModel")}
>
<span className="material-symbols-outlined text-sm">
{copied === `model-${model.id}` ? "check" : "content_copy"}
<div className="flex flex-col px-3 py-2 rounded-lg border border-border hover:bg-sidebar/50 min-w-[220px] max-w-md">
<div className="flex items-center gap-2 flex-wrap">
<span className="material-symbols-outlined text-base text-text-muted shrink-0">
smart_toy
</span>
</button>
<code className="text-xs text-text-muted font-mono bg-sidebar px-1.5 py-0.5 rounded">
{fullModel}
</code>
<button
onClick={() => onCopy(fullModel, `model-${model.id}`)}
className="p-0.5 hover:bg-sidebar rounded text-text-muted hover:text-primary"
title={t("copyModel")}
>
<span className="material-symbols-outlined text-sm">
{copied === `model-${model.id}` ? "check" : "content_copy"}
</span>
</button>
</div>
<ModelCompatPopover
t={t}
normalizeToolCallId={Boolean(normalizeToolCallId)}
preserveDeveloperRole={preserveDeveloperRole}
showDeveloperToggle={showDeveloperToggle}
onNormalizeChange={onNormalizeChange}
onPreserveChange={onPreserveChange}
disabled={compatDisabled}
/>
</div>
);
}
@@ -1311,6 +1533,13 @@ ModelRow.propTypes = {
alias: PropTypes.string,
copied: PropTypes.string,
onCopy: PropTypes.func.isRequired,
t: PropTypes.func,
showDeveloperToggle: PropTypes.bool,
normalizeToolCallId: PropTypes.bool,
preserveDeveloperRole: PropTypes.bool,
onNormalizeChange: PropTypes.func,
onPreserveChange: PropTypes.func,
compatDisabled: PropTypes.bool,
};
function PassthroughModelsSection({
@@ -1320,12 +1549,15 @@ function PassthroughModelsSection({
onCopy,
onSetAlias,
onDeleteAlias,
t,
effectiveModelNormalize,
effectiveModelPreserveDeveloper,
saveModelCompatFlags,
compatSavingModelId,
}) {
const t = useTranslations("providers");
const [newModel, setNewModel] = useState("");
const [adding, setAdding] = useState(false);
// Filter aliases for this provider - models are persisted via alias
const providerAliases = Object.entries(modelAliases).filter(([, model]: [string, any]) =>
(model as string).startsWith(`${providerAlias}/`)
);
@@ -1400,6 +1632,15 @@ function PassthroughModelsSection({
copied={copied}
onCopy={onCopy}
onDeleteAlias={() => onDeleteAlias(alias)}
t={t}
showDeveloperToggle
normalizeToolCallId={effectiveModelNormalize(modelId)}
preserveDeveloperRole={effectiveModelPreserveDeveloper(modelId)}
onNormalizeChange={(v) => saveModelCompatFlags(modelId, { normalizeToolCallId: v })}
onPreserveChange={(v) =>
saveModelCompatFlags(modelId, { preserveOpenAIDeveloperRole: v })
}
compatDisabled={compatSavingModelId === modelId}
/>
))}
</div>
@@ -1415,41 +1656,69 @@ PassthroughModelsSection.propTypes = {
onCopy: PropTypes.func.isRequired,
onSetAlias: PropTypes.func.isRequired,
onDeleteAlias: PropTypes.func.isRequired,
t: PropTypes.func.isRequired,
effectiveModelNormalize: PropTypes.func.isRequired,
effectiveModelPreserveDeveloper: PropTypes.func.isRequired,
saveModelCompatFlags: PropTypes.func.isRequired,
compatSavingModelId: PropTypes.string,
};
function PassthroughModelRow({ modelId, fullModel, copied, onCopy, onDeleteAlias }) {
const t = useTranslations("providers");
function PassthroughModelRow({
modelId,
fullModel,
copied,
onCopy,
onDeleteAlias,
t,
showDeveloperToggle = true,
normalizeToolCallId,
preserveDeveloperRole,
onNormalizeChange,
onPreserveChange,
compatDisabled,
}: any) {
return (
<div className="flex items-center gap-3 p-3 rounded-lg border border-border hover:bg-sidebar/50">
<span className="material-symbols-outlined text-base text-text-muted">smart_toy</span>
<div className="flex-1 min-w-0">
<p className="text-sm font-medium truncate">{modelId}</p>
<div className="flex items-center gap-1 mt-1">
<code className="text-xs text-text-muted font-mono bg-sidebar px-1.5 py-0.5 rounded">
{fullModel}
</code>
<button
onClick={() => onCopy(fullModel, `model-${modelId}`)}
className="p-0.5 hover:bg-sidebar rounded text-text-muted hover:text-primary"
title={t("copyModel")}
>
<span className="material-symbols-outlined text-sm">
{copied === `model-${modelId}` ? "check" : "content_copy"}
</span>
</button>
<div className="flex flex-col gap-0 p-3 rounded-lg border border-border hover:bg-sidebar/50">
<div className="flex items-start gap-3">
<span className="material-symbols-outlined text-base text-text-muted shrink-0">
smart_toy
</span>
<div className="flex-1 min-w-0">
<p className="text-sm font-medium truncate">{modelId}</p>
<div className="flex items-center gap-1 mt-1 flex-wrap">
<code className="text-xs text-text-muted font-mono bg-sidebar px-1.5 py-0.5 rounded">
{fullModel}
</code>
<button
onClick={() => onCopy(fullModel, `model-${modelId}`)}
className="p-0.5 hover:bg-sidebar rounded text-text-muted hover:text-primary"
title={t("copyModel")}
>
<span className="material-symbols-outlined text-sm">
{copied === `model-${modelId}` ? "check" : "content_copy"}
</span>
</button>
</div>
</div>
<button
onClick={onDeleteAlias}
className="p-1 hover:bg-red-50 rounded text-red-500 shrink-0"
title={t("removeModel")}
>
<span className="material-symbols-outlined text-sm">delete</span>
</button>
</div>
<div className="pl-9">
<ModelCompatPopover
t={t}
normalizeToolCallId={Boolean(normalizeToolCallId)}
preserveDeveloperRole={preserveDeveloperRole}
showDeveloperToggle={showDeveloperToggle}
onNormalizeChange={onNormalizeChange}
onPreserveChange={onPreserveChange}
disabled={compatDisabled}
/>
</div>
{/* Delete button */}
<button
onClick={onDeleteAlias}
className="p-1 hover:bg-red-50 rounded text-red-500"
title={t("removeModel")}
>
<span className="material-symbols-outlined text-sm">delete</span>
</button>
</div>
);
}
@@ -1460,11 +1729,18 @@ PassthroughModelRow.propTypes = {
copied: PropTypes.string,
onCopy: PropTypes.func.isRequired,
onDeleteAlias: PropTypes.func.isRequired,
t: PropTypes.func,
showDeveloperToggle: PropTypes.bool,
normalizeToolCallId: PropTypes.bool,
preserveDeveloperRole: PropTypes.bool,
onNormalizeChange: PropTypes.func,
onPreserveChange: PropTypes.func,
compatDisabled: PropTypes.bool,
};
// ============ Custom Models Section (for ALL providers) ============
function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
function CustomModelsSection({ providerId, providerAlias, copied, onCopy, onModelsChanged }) {
const t = useTranslations("providers");
const notify = useNotificationStore();
const [customModels, setCustomModels] = useState([]);
@@ -1478,6 +1754,7 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
const [editingApiFormat, setEditingApiFormat] = useState("chat-completions");
const [editingEndpoints, setEditingEndpoints] = useState<string[]>(["chat"]);
const [editingNormalizeToolCallId, setEditingNormalizeToolCallId] = useState(false);
const [editingPreserveDeveloperRole, setEditingPreserveDeveloperRole] = useState(false);
const [savingModelId, setSavingModelId] = useState<string | null>(null);
const fetchCustomModels = useCallback(async () => {
@@ -1519,6 +1796,7 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
setNewApiFormat("chat-completions");
setNewEndpoints(["chat"]);
await fetchCustomModels();
onModelsChanged?.();
}
} catch (e) {
console.error("Failed to add custom model:", e);
@@ -1536,6 +1814,7 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
}
);
await fetchCustomModels();
onModelsChanged?.();
} catch (e) {
console.error("Failed to remove custom model:", e);
}
@@ -1550,6 +1829,11 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
: ["chat"]
);
setEditingNormalizeToolCallId(Boolean(model.normalizeToolCallId));
setEditingPreserveDeveloperRole(
Object.prototype.hasOwnProperty.call(model, "preserveOpenAIDeveloperRole")
? Boolean(model.preserveOpenAIDeveloperRole)
: true
);
};
const cancelEdit = () => {
@@ -1557,6 +1841,7 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
setEditingApiFormat("chat-completions");
setEditingEndpoints(["chat"]);
setEditingNormalizeToolCallId(false);
setEditingPreserveDeveloperRole(true);
setSavingModelId(null);
};
@@ -1581,6 +1866,7 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
apiFormat: editingApiFormat,
supportedEndpoints: editingEndpoints,
normalizeToolCallId: editingNormalizeToolCallId,
preserveOpenAIDeveloperRole: editingPreserveDeveloperRole,
}),
});
@@ -1589,6 +1875,7 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
}
await fetchCustomModels();
onModelsChanged?.();
notify.success("Saved model endpoint settings");
cancelEdit();
} catch (e) {
@@ -1745,11 +2032,19 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
{model.normalizeToolCallId && (
<span
className="text-[10px] px-1.5 py-0.5 rounded-full bg-slate-500/15 text-slate-400 font-medium"
title="9-char tool call ID (Mistral)"
title={t("normalizeToolCallIdLabel")}
>
ID×9
</span>
)}
{model.preserveOpenAIDeveloperRole === false && (
<span
className="text-[10px] px-1.5 py-0.5 rounded-full bg-cyan-500/15 text-cyan-400 font-medium"
title={t("compatDoNotPreserveDeveloper")}
>
{t("compatBadgeNoPreserve")}
</span>
)}
</div>
{editingModelId === model.id && (
@@ -1802,16 +2097,17 @@ function CustomModelsSection({ providerId, providerAlias, copied, onCopy }) {
))}
</div>
</div>
<label className="flex items-center gap-2 text-xs text-text-main cursor-pointer">
<input
type="checkbox"
checked={editingNormalizeToolCallId}
onChange={(e) => setEditingNormalizeToolCallId(e.target.checked)}
className="rounded border-border"
/>
Normalize Tool Call ID (9 chars, Mistral)
</label>
</div>
<div className="mt-3 pt-3 border-t border-border/80 w-full">
<ModelCompatPopover
t={t}
normalizeToolCallId={editingNormalizeToolCallId}
preserveDeveloperRole={editingPreserveDeveloperRole}
showDeveloperToggle
onNormalizeChange={setEditingNormalizeToolCallId}
onPreserveChange={setEditingPreserveDeveloperRole}
disabled={savingModelId === model.id}
/>
</div>
<div className="mt-3 flex items-center gap-2">
<Button
@@ -1860,6 +2156,7 @@ CustomModelsSection.propTypes = {
providerAlias: PropTypes.string.isRequired,
copied: PropTypes.string,
onCopy: PropTypes.func.isRequired,
onModelsChanged: PropTypes.func,
};
function CompatibleModelsSection({
@@ -1873,8 +2170,13 @@ function CompatibleModelsSection({
connections,
isAnthropic,
onImportWithProgress,
t,
effectiveModelNormalize,
effectiveModelPreserveDeveloper,
saveModelCompatFlags,
compatSavingModelId,
onModelsChanged,
}) {
const t = useTranslations("providers");
const [newModel, setNewModel] = useState("");
const [adding, setAdding] = useState(false);
const [importing, setImporting] = useState(false);
@@ -1940,6 +2242,7 @@ function CompatibleModelsSection({
await onSetAlias(modelId, resolvedAlias, providerStorageAlias);
setNewModel("");
notify.success(t("modelAddedSuccess", { modelId }));
onModelsChanged?.();
} catch (error) {
console.error("Error adding model:", error);
notify.error(error instanceof Error ? error.message : t("failedAddModelTryAgain"));
@@ -2016,6 +2319,7 @@ function CompatibleModelsSection({
// Also delete the alias
await onDeleteAlias(alias);
notify.success(t("modelRemovedSuccess"));
onModelsChanged?.();
} catch (error) {
console.error("Error deleting model:", error);
notify.error(error instanceof Error ? error.message : t("failedDeleteModelTryAgain"));
@@ -2078,6 +2382,15 @@ function CompatibleModelsSection({
copied={copied}
onCopy={onCopy}
onDeleteAlias={() => handleDeleteModel(modelId, alias)}
t={t}
showDeveloperToggle={!isAnthropic}
normalizeToolCallId={effectiveModelNormalize(modelId)}
preserveDeveloperRole={effectiveModelPreserveDeveloper(modelId)}
onNormalizeChange={(v) => saveModelCompatFlags(modelId, { normalizeToolCallId: v })}
onPreserveChange={(v) =>
saveModelCompatFlags(modelId, { preserveOpenAIDeveloperRole: v })
}
compatDisabled={compatSavingModelId === modelId}
/>
))}
</div>
@@ -2102,6 +2415,12 @@ CompatibleModelsSection.propTypes = {
).isRequired,
isAnthropic: PropTypes.bool,
onImportWithProgress: PropTypes.func.isRequired,
t: PropTypes.func.isRequired,
effectiveModelNormalize: PropTypes.func.isRequired,
effectiveModelPreserveDeveloper: PropTypes.func.isRequired,
saveModelCompatFlags: PropTypes.func.isRequired,
compatSavingModelId: PropTypes.string,
onModelsChanged: PropTypes.func,
};
function CooldownTimer({ until }) {
+47 -6
View File
@@ -4,6 +4,8 @@ import {
addCustomModel,
removeCustomModel,
updateCustomModel,
getModelCompatOverrides,
mergeModelCompatOverride,
} from "@/lib/localDb";
import { isAuthenticated } from "@/shared/utils/apiAuth";
import { providerModelMutationSchema } from "@/shared/validation/schemas";
@@ -27,8 +29,9 @@ export async function GET(request) {
const provider = searchParams.get("provider");
const models = provider ? await getCustomModels(provider) : await getAllCustomModels();
const modelCompatOverrides = provider ? getModelCompatOverrides(provider) : [];
return Response.json({ models });
return Response.json({ models, modelCompatOverrides });
} catch (error) {
return Response.json(
{ error: { message: error.message, type: "server_error" } },
@@ -113,17 +116,55 @@ export async function PUT(request) {
return Response.json({ error: validation.error }, { status: 400 });
}
const { provider, modelId, modelName, apiFormat, supportedEndpoints, normalizeToolCallId } =
validation.data;
const model = await updateCustomModel(provider, modelId, {
const {
provider,
modelId,
modelName,
apiFormat,
supportedEndpoints,
normalizeToolCallId,
});
preserveOpenAIDeveloperRole,
} = validation.data;
const raw = rawBody as Record<string, unknown>;
const updates: Record<string, unknown> = {};
if ("modelName" in raw) updates.modelName = modelName;
if ("apiFormat" in raw) updates.apiFormat = apiFormat;
if ("supportedEndpoints" in raw) updates.supportedEndpoints = supportedEndpoints;
if ("normalizeToolCallId" in raw) updates.normalizeToolCallId = normalizeToolCallId;
if ("preserveOpenAIDeveloperRole" in raw)
updates.preserveOpenAIDeveloperRole = preserveOpenAIDeveloperRole;
const model = await updateCustomModel(provider, modelId, updates);
if (!model) {
const rawKeys = Object.keys(raw);
const compatOnly =
rawKeys.length > 0 &&
rawKeys.every((k) =>
["provider", "modelId", "normalizeToolCallId", "preserveOpenAIDeveloperRole"].includes(k)
) &&
("normalizeToolCallId" in raw || "preserveOpenAIDeveloperRole" in raw);
if (compatOnly) {
const patch: {
normalizeToolCallId?: boolean;
preserveOpenAIDeveloperRole?: boolean;
} = {};
if ("normalizeToolCallId" in raw && typeof normalizeToolCallId === "boolean") {
patch.normalizeToolCallId = normalizeToolCallId;
}
if (
"preserveOpenAIDeveloperRole" in raw &&
typeof preserveOpenAIDeveloperRole === "boolean"
) {
patch.preserveOpenAIDeveloperRole = preserveOpenAIDeveloperRole;
}
mergeModelCompatOverride(provider, modelId, patch);
return Response.json({
ok: true,
modelCompatOverrides: getModelCompatOverrides(provider),
});
}
return Response.json(
{ error: { message: "Model not found", type: "not_found" } },
{ status: 404 }
+33 -3
View File
@@ -12,6 +12,7 @@ import {
getEmbeddingProvider,
buildDynamicEmbeddingProvider,
type EmbeddingProviderNodeRow,
type EmbeddingProvider,
} from "@omniroute/open-sse/config/embeddingRegistry.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
@@ -116,9 +117,9 @@ export async function POST(request) {
// Load local provider_nodes for embedding routing (only localhost — prevents auth bypass/SSRF)
let dynamicProviders: ReturnType<typeof buildDynamicEmbeddingProvider>[] = [];
try {
const nodes = await getProviderNodes();
const nodes = (await getProviderNodes()) as unknown as EmbeddingProviderNodeRow[];
dynamicProviders = (Array.isArray(nodes) ? nodes : [])
.filter((n: EmbeddingProviderNodeRow) => {
.filter((n) => {
// provider_nodes apiType is "chat" or "responses" (not "embeddings") — local OpenAI-compatible
// backends expose /embeddings under the same base URL as chat, so we build the URL as baseUrl + /embeddings.
if (n.apiType !== "chat" && n.apiType !== "responses") return false;
@@ -157,9 +158,38 @@ export async function POST(request) {
}
// Resolve provider config — dynamic first (local override), then hardcoded
const providerConfig =
let providerConfig: EmbeddingProvider | null =
dynamicProviders.find((dp) => dp.id === provider) || getEmbeddingProvider(provider) || null;
// #496: Fallback — resolve from ALL provider_nodes (not just localhost)
// This enables custom embedding models (e.g. google/gemini-embedding-001) whose
// providers have remote baseUrls. Safe because getProviderCredentials() authenticates.
if (!providerConfig) {
try {
const allNodes = (await getProviderNodes()) as unknown as EmbeddingProviderNodeRow[];
const matchingNode = (Array.isArray(allNodes) ? allNodes : []).find(
(n) =>
n.prefix === provider && (n.apiType === "chat" || n.apiType === "responses") && n.baseUrl
);
if (matchingNode) {
const baseUrl = String(matchingNode.baseUrl).replace(/\/+$/, "");
providerConfig = {
id: matchingNode.prefix,
baseUrl: `${baseUrl}/embeddings`,
authType: "apikey",
authHeader: "bearer",
models: [],
};
log.info(
"EMBED",
`Resolved custom embedding provider: ${provider}${providerConfig.baseUrl}`
);
}
} catch (err) {
log.error("EMBED", `Failed to resolve custom embedding provider ${provider}: ${err}`);
}
}
if (!providerConfig) {
return errorResponse(
HTTP_STATUS.BAD_REQUEST,
+10
View File
@@ -1382,6 +1382,8 @@
"addFirstConnectionHint": "Add your first connection to get started",
"addConnection": "Add Connection",
"availableModels": "Available Models",
"builtInModels": "Built-in models",
"builtInModelsHint": "Registry models for this provider. Use the pencil to set compatibility options.",
"pageAutoRefresh": "Page will refresh automatically...",
"statusDisabled": "disabled",
"statusConnected": "connected",
@@ -1422,6 +1424,14 @@
"openRouterModelPlaceholder": "anthropic/claude-3-opus",
"customModels": "Custom Models",
"customModelsHint": "Add model IDs not in the default list. These will be available for routing.",
"normalizeToolCallIdLabel": "Normalize tool call IDs to 9 characters (e.g. Mistral)",
"preserveDeveloperRoleLabel": "Keep OpenAI Responses developer role (do not map to system)",
"compatAdjustmentsTitle": "Compatibility",
"compatButtonLabel": "Compatibility",
"compatToolIdShort": "Tool ID 9",
"compatDeveloperShort": "Developer role",
"compatDoNotPreserveDeveloper": "Do not preserve developer role",
"compatBadgeNoPreserve": "No preserve",
"modelId": "Model ID",
"customModelPlaceholder": "e.g. gpt-4.5-turbo",
"loading": "Loading...",
+10
View File
@@ -1382,6 +1382,8 @@
"addFirstConnectionHint": "添加您的第一个连接以开始使用",
"addConnection": "添加连接",
"availableModels": "可用模型",
"builtInModels": "内置模型",
"builtInModelsHint": "该提供商的注册表模型。点击铅笔可设置兼容选项。",
"pageAutoRefresh": "页面会自动刷新...",
"statusDisabled": "已禁用",
"statusConnected": "已连接",
@@ -1422,6 +1424,14 @@
"openRouterModelPlaceholder": "anthropic/claude-3-opus",
"customModels": "自定义模型",
"customModelsHint": "添加默认列表中没有的模型 ID,这些模型也能参与路由。",
"normalizeToolCallIdLabel": "将工具调用 ID 规范为 9 位(如 Mistral",
"preserveDeveloperRoleLabel": "保留 Responses 的 developer 角色(不映射为 system",
"compatAdjustmentsTitle": "兼容性",
"compatButtonLabel": "兼容性",
"compatToolIdShort": "工具 ID 9 位",
"compatDeveloperShort": "Developer 角色",
"compatDoNotPreserveDeveloper": "不保留 developer 角色",
"compatBadgeNoPreserve": "不保留",
"modelId": "模型 ID",
"customModelPlaceholder": "例如:gpt-4.5-turbo",
"loading": "正在加载...",
+130 -13
View File
@@ -7,6 +7,82 @@ import { backupDbFile } from "./backup";
type JsonRecord = Record<string, unknown>;
/** Built-in / alias models: tool-call + developer-role flags without a full custom row */
const MODEL_COMPAT_NAMESPACE = "modelCompatOverrides";
export type ModelCompatOverride = {
id: string;
normalizeToolCallId?: boolean;
preserveOpenAIDeveloperRole?: boolean;
};
function readCompatList(providerId: string): ModelCompatOverride[] {
const db = getDbInstance();
const row = db
.prepare("SELECT value FROM key_value WHERE namespace = ? AND key = ?")
.get(MODEL_COMPAT_NAMESPACE, providerId);
const value = getKeyValue(row).value;
if (!value) return [];
try {
const parsed = JSON.parse(value);
return Array.isArray(parsed) ? parsed : [];
} catch {
return [];
}
}
function writeCompatList(providerId: string, list: ModelCompatOverride[]) {
const db = getDbInstance();
if (list.length === 0) {
db.prepare("DELETE FROM key_value WHERE namespace = ? AND key = ?").run(
MODEL_COMPAT_NAMESPACE,
providerId
);
} else {
db.prepare("INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES (?, ?, ?)").run(
MODEL_COMPAT_NAMESPACE,
providerId,
JSON.stringify(list)
);
}
backupDbFile("pre-write");
}
export function getModelCompatOverrides(providerId: string): ModelCompatOverride[] {
return readCompatList(providerId);
}
export function mergeModelCompatOverride(
providerId: string,
modelId: string,
patch: Partial<Pick<ModelCompatOverride, "normalizeToolCallId" | "preserveOpenAIDeveloperRole">>
) {
const list = readCompatList(providerId);
const idx = list.findIndex((e) => e.id === modelId);
const prev = idx >= 0 ? { ...list[idx] } : { id: modelId };
const next: ModelCompatOverride = { ...prev, id: modelId };
if ("normalizeToolCallId" in patch) {
if (patch.normalizeToolCallId) next.normalizeToolCallId = true;
else delete next.normalizeToolCallId;
}
if ("preserveOpenAIDeveloperRole" in patch) {
next.preserveOpenAIDeveloperRole = Boolean(patch.preserveOpenAIDeveloperRole);
}
const filtered = list.filter((e) => e.id !== modelId);
const hasPreserveFlag = Object.prototype.hasOwnProperty.call(next, "preserveOpenAIDeveloperRole");
if (next.normalizeToolCallId || hasPreserveFlag) {
filtered.push(next);
}
writeCompatList(providerId, filtered);
}
export function removeModelCompatOverride(providerId: string, modelId: string) {
const list = readCompatList(providerId);
const filtered = list.filter((e) => e.id !== modelId);
if (filtered.length === list.length) return;
writeCompatList(providerId, filtered);
}
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
@@ -174,11 +250,16 @@ export async function removeCustomModel(providerId, modelId) {
);
}
removeModelCompatOverride(providerId, modelId);
backupDbFile("pre-write");
return true;
}
export async function updateCustomModel(providerId, modelId, updates = {}) {
export async function updateCustomModel(
providerId: string,
modelId: string,
updates: Record<string, unknown> = {}
) {
const db = getDbInstance();
const row = db
.prepare("SELECT value FROM key_value WHERE namespace = 'customModels' AND key = ?")
@@ -203,6 +284,9 @@ export async function updateCustomModel(providerId, modelId, updates = {}) {
...(updates.normalizeToolCallId !== undefined
? { normalizeToolCallId: Boolean(updates.normalizeToolCallId) }
: {}),
...(updates.preserveOpenAIDeveloperRole !== undefined
? { preserveOpenAIDeveloperRole: Boolean(updates.preserveOpenAIDeveloperRole) }
: {}),
};
models[index] = next;
@@ -216,24 +300,57 @@ export async function updateCustomModel(providerId, modelId, updates = {}) {
return next;
}
/**
* Whether the given provider/model has "normalize tool call id" (9-char Mistral-style) enabled.
* Only custom models can have this set; returns false for built-in models.
*/
export function getModelNormalizeToolCallId(providerId: string, modelId: string): boolean {
/** Single custom model row from key_value customModels, or null */
function getCustomModelRow(providerId: string, modelId: string): JsonRecord | null {
const db = getDbInstance();
const row = db
.prepare("SELECT value FROM key_value WHERE namespace = 'customModels' AND key = ?")
.get(providerId);
const value = getKeyValue(row).value;
if (!value) return false;
let models: { id: string; normalizeToolCallId?: boolean }[];
if (!value) return null;
try {
models = JSON.parse(value);
const models = JSON.parse(value) as unknown;
if (!Array.isArray(models)) return null;
const m = models.find((x: unknown) => {
if (!x || typeof x !== "object" || Array.isArray(x)) return false;
return (x as { id?: string }).id === modelId;
}) as JsonRecord | undefined;
return m ?? null;
} catch {
return false;
return null;
}
if (!Array.isArray(models)) return false;
const m = models.find((x: { id: string }) => x.id === modelId);
return Boolean(m?.normalizeToolCallId);
}
/**
* Whether the given provider/model has "normalize tool call id" (9-char Mistral-style) enabled.
* Custom model row wins; otherwise {@link getModelCompatOverrides}.
*/
export function getModelNormalizeToolCallId(providerId: string, modelId: string): boolean {
const m = getCustomModelRow(providerId, modelId);
if (m) return Boolean(m.normalizeToolCallId);
const co = readCompatList(providerId).find((e) => e.id === modelId);
return Boolean(co?.normalizeToolCallId);
}
/**
* Explicit preserve-openai-developer preference for this provider/model.
* `undefined` = unset routing keeps legacy default (preserve developer for OpenAI format).
* `false` = map developer system (e.g. MiniMax). `true` = keep developer.
*/
export function getModelPreserveOpenAIDeveloperRole(
providerId: string,
modelId: string
): boolean | undefined {
const m = getCustomModelRow(providerId, modelId);
if (m) {
if (Object.prototype.hasOwnProperty.call(m, "preserveOpenAIDeveloperRole")) {
return Boolean(m.preserveOpenAIDeveloperRole);
}
return undefined;
}
const co = readCompatList(providerId).find((e) => e.id === modelId);
if (co && Object.prototype.hasOwnProperty.call(co, "preserveOpenAIDeveloperRole")) {
return Boolean(co.preserveOpenAIDeveloperRole);
}
return undefined;
}
+5
View File
@@ -41,6 +41,11 @@ export {
addCustomModel,
removeCustomModel,
updateCustomModel,
getModelCompatOverrides,
mergeModelCompatOverride,
removeModelCompatOverride,
getModelNormalizeToolCallId,
getModelPreserveOpenAIDeveloperRole,
} from "./db/models";
export {
+1
View File
@@ -348,6 +348,7 @@ export const providerModelMutationSchema = z.object({
apiFormat: z.enum(["chat-completions", "responses"]).default("chat-completions"),
supportedEndpoints: z.array(z.enum(["chat", "embeddings", "images", "audio"])).default(["chat"]),
normalizeToolCallId: z.boolean().optional(),
preserveOpenAIDeveloperRole: z.boolean().optional(),
});
const pricingFieldsSchema = z