feat(xai): support grok-4.5 reasoning effort mapping (#519)

xAI's Chat Completions API takes reasoning effort as a flat top-level
reasoning_effort string (grok-4.5: low/medium/high, default high), while
GoModel's normalized shape is the nested reasoning.effort object. Add an
AdaptChatRequest hook to the xai provider (same pattern as deepseek and
gemini) that rewrites the nested shape into the flat field, downgrading
xhigh/max to high except on the multi-agent Grok family, which accepts
xhigh natively. The Responses API keeps the nested shape untouched since
it is xAI-native there.

Also add grok-4.5 to the xai contract fixture, document the provider in
docs/providers/xai.mdx (reasoning mapping table, prompt-cache affinity
via X-Grok-Conv-Id / prompt_cache_key), bump the overview example, and
apply a go fix modernization in keyring_test.go to keep the fix-check
guard green.

Verified end-to-end against the live xAI API: chat, vision, streamed
multimodal tool calls, tool round-trip, native and streamed /responses,
bare-name routing, per-model cost accounting, and prompt-cache affinity
(derived conv-id stable across turns, upstream cache hits confirmed).

Co-authored-by: Claude Fable 5 <[email protected]>

fix(realtime): resolve virtual-model aliases on realtime and audio en…

…dpoints (#514)

Aliases 404'd on every realtime and audio endpoint while resolving fine on
chat, responses, and embeddings:

  POST /v1/realtime/calls?model=<alias>            -> 404 model not found
  POST /v1/realtime/client_secrets (session.model) -> 404
  GET  /v1/realtime?model=<alias>                  -> 404
  POST /v1/audio/speech                            -> 404
  POST /v1/audio/transcriptions                    -> 404

Two causes. realtimeService and audioService never received the virtual-model
resolver (h.modelResolver), so prepare() only type-asserted the registry-only
selectorResolver on the Router, which canonicalizes concrete ids and leaves an
alias untouched. And even the parsed selector was discarded: the raw request
model was forwarded to the router, which then failed the provider lookup. These
endpoints resolve their model in the service layer because OperationRealtime and
the audio operations are absent from the workflow middleware's resolveWorkflow
switch, so ensureRequestModelResolution never runs for them.

Both services now resolve through the shared resolveServiceModel helper, which
runs the same two-step gateway.ResolveExecutionSelector (alias table, then
provider registry) used by chat/responses/embeddings, and forward the resolved
concrete selector upstream.

The prior unit test could not catch this: it injected a mock whose ResolveModel
faked the alias resolution the production Router never performs, and asserted the
router received the raw alias under a comment claiming the opposite. Replaced with
regression tests that drive a real virtualmodels.Service through a registry-only
provider double for all three realtime routes plus speech and transcription; each
fails if the resolver is unwired or the raw model is forwarded.

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>

feat(ratelimit): scoped request, token, and concurrency limits (user …

…paths, providers, models) (#482)

* feat(ratelimit): per-user-path request, token, and concurrency limits

Add a rate limiting feature that caps how fast a user_path subtree can
consume the gateway, complementing budgets (spend control) with traffic
control:

- Rules cap max_requests and/or max_tokens per minute/hour/day/custom
  window, plus a "concurrent" period for in-flight request caps. One
  shared counter per rule covers its whole subtree; per-key limits fall
  out of binding each managed key to its own user_path.
- Enforcement runs before the budget check on every model endpoint
  (chat, responses, messages, embeddings, audio, passthrough, realtime
  sessions, batch submission). Requests use an in-memory sliding-window
  counter; tokens are post-accounted from usage entries via a usage
  logger tap (cache hits excluded, requires USAGE_ENABLED); realtime
  sessions hold their concurrency slot for the session lifetime.
- Breaches return 429 with an OpenAI-shaped error (code
  rate_limit_exceeded) and an accurate Retry-After; successes carry
  x-ratelimit-{limit,remaining,reset}-{requests,tokens} headers from
  the most-constrained matching rule.
- Managed via a new dashboard Rate Limits page, /admin/rate-limits
  CRUD + reset endpoints, a rate_limits: YAML block, and
  SET_RATE_LIMIT_<PATH> env vars (rpm/tpm/rph/tph/rpd/tpd/concurrent).
  Gated by RATE_LIMITS_ENABLED (default true; no-op without rules).
- Counters are per instance and reset on restart (documented); rule
  definitions persist in a new rate_limits store on all three backends.
- The SET_BUDGET_*/SET_RATE_LIMIT_* env appliers share a new generic
  per-user-path env merge helper instead of duplicating the loop.

Spec with gateway research (LiteLLM, Bifrost, Portkey, Kong,
Cloudflare, Helicone) in docs/dev/2026-07-05_rate-limiting-spec.md;
user docs in docs/features/rate-limits.mdx.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(ratelimit): address review findings

- Retry-After now reports the earliest second the sliding-window
  estimate would actually admit a retry, instead of the next bucket
  boundary, which under-promised right after a rollover while the
  previous window still weighed in (docs updated to match).
- The Mongo store's upsert is source-aware like the SQL stores: a
  config-sourced write only updates config rows, and the duplicate-key
  insert produced by a shadowing manual row is treated as the intended
  precedence, not a failure.
- Guardrail LLM calls run through the tapped usage logger so their
  tokens count toward rate limit token windows.
- The dashboard form rejects blank custom period seconds instead of
  coercing them to 0 and silently submitting a concurrent rule.
- Deduped the period_seconds validation shared by DeleteRule and the
  stores, and the period constants shared by the env-name and YAML
  period parsers.
- Added an e2e concurrency test (held in-flight request blocks the
  next one, slot frees on completion), a batch-enforcer test with a
  live limiter, exact Retry-After recovery tests, and unit coverage
  for the Mongo duplicate-key classification; removed a redundant
  global.fetch override from the dashboard module test.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(ratelimit): scope rules to user paths, providers, and models

Rule identity becomes (scope, subject, period). user_path rules keep their
admission-control behavior (429 + Retry-After + x-ratelimit-* headers).
Provider and model rules act as routing constraints: virtual-model load
balancing consults a rate-limit-aware catalog and failover sweeps consult a
RouteGate, so saturated targets are skipped like stale-inventory providers
and clients only see 429 when no viable target remains. Token windows are
charged to the executed provider/model from usage entries, keeping accounting
correct under aliasing and failover.

Config gains rate_limits.providers / rate_limits.models blocks and
SET_PROVIDER_RATE_LIMIT_<NAME> env vars (suffix underscores map to hyphens
like provider-instance env vars; model rules are YAML/admin-only). The
dashboard form gets a scope selector and rows show scope chips. Pre-scope
rate_limits tables/documents migrate in place at store init on SQLite,
PostgreSQL, and MongoDB.

Also fixes the rate-limit modal silently refusing to save concurrent rules:
native form validation blocked submission on the hidden period-seconds input
(min=1 vs value 0); the form is now novalidate and scope/period switches
clear stale hidden fields.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(ratelimit): address scoped-rules review findings

- Run the pre-scope schema migrations in one transaction on SQLite and
  PostgreSQL: a crash mid-rebuild could leave the table renamed away, and
  the next startup would create a fresh empty rate_limits and orphan every
  rule. The table DDL is now a single shared constant per store.
- Lowercase model rule subjects at normalization so storage matches the
  case-insensitive matching (no duplicate OpenAI/GPT-4o vs openai/gpt-4o
  rules covering the same requests).
- Report the exceeded rule with the longest recovery when several windows
  are exhausted at once, keeping Retry-After honest in one round trip; the
  breach check is now shared between admission and the read-only route
  probe.
- Restrict Mongo's benign duplicate-key skip to config-sourced writes and
  retry manual insert races as plain updates (last write wins, matching
  the SQL stores).
- Reject conflicting subject + user_path on provider/model admin requests
  instead of silently dropping the user_path.
- Pin in-memory SQLite test databases to one connection, cover the stream
  failover route-gate skip, and extract the shared rate-limit + budget
  admission sequence used by five dispatch sites.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(ratelimit): surface the retry error on Mongo duplicate-key recovery

When the retry after a manual insert race fails for a different reason,
wrap that failure instead of the original duplicate-key error, so the
real cause is diagnosable.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(dashboard): rate-limit inspector on Models page and movable rules

Every model row and provider group header on the Models page gets a gauge
button that opens an effective-limits inspector: the model's own rules, the
provider rules it shares, and the global root user-path rules, each with
live usage and add/edit actions. The editor opened from the inspector
returns to it on close.

The rate-limit editor's scope, subject, and period are now editable while
editing. Changing any of them moves the rule: the new key is created first
and the old rule deleted after, so a failed delete can never lose the rule,
and an edit that merely respells the same identity (case, slashes) is
detected via a client-side mirror of the server normalization and treated
as an in-place update. A hint in the form explains that moving a rule
restarts its live counters.

Co-Authored-By: Claude Fable 5 <[email protected]>

* feat(dashboard): usage-tinted inspector rows and gauge limit indicators

The rate-limit inspector rows now double as usage meters: the row
background fills left-to-right with the share of the most constrained cap
used, ramping green to warning at 75% and danger at 100%, with the exact
percentage in the row tooltip.

The Models page gauge buttons carry the limit state: fully painted (with
the active dot) when the model or provider has its own rules, half painted
when only provider or global root rules throttle it, plain when nothing
applies. Tooltips spell the state out, and the models page now loads the
rule list so the indicators are correct on first render.

Co-Authored-By: Claude Fable 5 <[email protected]>

* style(dashboard): add vertical padding to budget-list

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(ratelimit): let a saturated primary route defer to configured failover

Admission previously returned 429 for provider/model-scoped breaches before
dispatch ever ran, so the failover sweep (and its RouteGate) never got the
chance to serve the request from another target — breaking the advertised
"429 only when no viable target remains" behavior for direct routes with
failover rules.

The primary provider still must not be called (it would happily serve the
request and defeat the gateway's limit), so the breach becomes a synthetic
primary failure instead: when failover targets exist, admission re-admits
the request against its consumer limits only, stamps the stored 429 into
the context, and dispatch skips the primary call and enters the failover
sweep seeded with that error — for both response and stream paths. The
sweep still skips rate-saturated candidates, and the original 429 with its
Retry-After surfaces unchanged when nothing can take the request.

Consumer (user-path) breaches never defer — switching targets cannot
relieve them — and embeddings keep rejecting outright since their failover
is disabled. The exceeded rule now travels in the gateway error's unwrap
chain (never serialized) so admission can inspect its scope.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(dashboard): address review findings on the rate-limit inspector round

- Memoize the Models-page gauge indicator state per rules-list generation:
  Alpine evaluates the class and label bindings several times per row on
  every render, each previously rescanning the full rules list.
- Count the Models-page rate-limit editor and inspector in
  overlayDialogOpen() so the modal-open treatment applies there too.
- Label the rate-limit editor dialog via aria-labelledby on its heading so
  assistive tech announces the Edit/Create title the UI shows.
- Extract the Mongo bulk-upsert error decision into classifyBulkWriteError
  and pin its contract (success, config-shadowing skip, manual-race retry,
  real failure) with a table test.

Skipped: replacing the JSON.stringify assertions with assert.deepEqual —
the tests use node:assert/strict where deepEqual is prototype-sensitive,
and vm.runInContext objects carry foreign prototypes, so the suggested
change fails exactly the way these tests originally did.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>

tests(e2e): added new labelling related tests to the set

tests: better test coverage for failover (#450)

fix(usage): apply cached-token discounts to all OpenAI-compatible pro…

…viders (#439)

Cost calculation only applied per-token-type pricing (cached input,
reasoning, audio) for six provider types listed in providerMappings
(openai, openrouter, anthropic, gemini, groq, xai). Every other
OpenAI-compatible provider — xiaomi, deepseek, zai, minimax, bailian,
oracle, azure, vllm, ollama, opencode_go — got no mapping, so their
cached prompt tokens were billed at the full input rate instead of the
much cheaper cached-input rate, over-charging cache-heavy workloads.

This is the cause of issue #435: Xiaomi MiMo bills cache-hit input at
~1/50th of cache-miss input, but the gateway charged every input token
at the full rate, reporting roughly 4x the real cost.

Default any provider not in providerMappings to the OpenAI-compatible
token mappings, since they all speak the OpenAI usage schema. Also map
DeepSeek's non-standard top-level cache fields: prompt_cache_hit_tokens
(cached-input alias; the hit count is already inside prompt_tokens) and
prompt_cache_miss_tokens (informational remainder, no separate rate).

Note: the discount still requires the model's registry pricing to carry
cached_input_per_mtok; models without it (e.g. mimo-v2.5) need a
companion ai-model-list data update.

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>

test(e2e): make S128 throughput scenario deterministic (#438)

S128 ("token throughput reflects live traffic") compared the trailing two
minute-buckets before and after a request and asserted the sum increased. That is
racy: under heavy preceding traffic, when a minute boundary is crossed between the
two reads, an earlier high-traffic bucket rolls out of the trailing 2-bucket
window, so the "after" sum can be *lower* than "before" (observed: before=9340
after=606) and the scenario fails despite the feature working.

Replace the before/after delta with a deterministic check: after the request has
flushed to the usage store, assert the two most recent minute buckets are
non-empty. A just-flushed request is < 60s old, so its tokens must land in one of
those buckets — no window-roll race.

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>

docs: rename aliases to virtual models and regenerate API reference (#…

…427)

* docs: rename aliases to virtual models and regenerate API reference

Aliases and access overrides were unified into "virtual models"
(ADR-0008, #423/#424). Bring the user-facing docs and the generated API
reference in line, and move the feature page to its new URL.

- Rewrite the feature page as Virtual Models (redirects/aliases plus
  access policies); move /features/aliases -> /features/virtual-models
  with a docs.json redirect and rename the screenshot alongside it.
- Update related terminology in user-path, audio, anthropic-messages,
  responses, and azure docs.
- Regenerate docs/openapi.json and cmd/gomodel/docs/docs.go:
  /admin/model-overrides -> /admin/virtual-models, and
  modeloverrides.* -> virtualmodels.* schemas.
- Unbreak swagger generation: add swaggertype tags to the goccy/go-json
  json.RawMessage fields in internal/anthropicapi (matching the
  internal/core convention) and point tools/openapi-postprocess.mjs at
  the new virtual-models admin schemas and endpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* test(e2e): use /admin/virtual-models in release e2e scenarios

Migrate the release e2e alias scenarios to the unified virtual-models
admin endpoint and its request/response shape (source instead of name,
provider-qualified target_model, redirect kind/targets assertions),
following the aliases -> virtual models change.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* docs(admin): document 502 on virtual-models writes and align Swagger 2 spec

Address PR review feedback:

- /admin/virtual-models PUT and DELETE return 502 (StatusBadGateway) on store
  write failures via virtualModelWriteError, but the annotations omitted it
  (PUT documented 500, which the handler never returns). Document 502 on both,
  matching the model-pricing-overrides handler.
- Mirror the virtual-models required-field and array bounds into
  swagger-postprocess.mjs so the embedded Swagger 2 spec (cmd/gomodel/docs/docs.go)
  matches the published OpenAPI: source required on upsert/delete, user_paths
  maxItems, and the /admin/virtual-models listing maxItems cap.

Regenerated docs/openapi.json and cmd/gomodel/docs/docs.go.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>

perf: faster model resolution, JSON decoding, and request snapshotting (

#413)

* perf: faster model resolution, JSON decoding, and request snapshotting

Reduce per-request CPU and allocations on the gateway hot path. Changes are
behavior-preserving for all valid input; the one intentional difference is
documented and tested.

Model resolution (O(1)):
- Add a lazy provider-selector index to the registry (qualifiedByName /
  qualifiedByType), invalidated at the existing single cache-invalidation point.
- Route qualified-selector resolution through it via an optional
  qualifiedSelectorResolver interface, with the catalog scan kept as a fallback
  for non-indexed lookups and raw slash-shaped model IDs.
- Resolution is now O(1) and constant in catalog size (was O(N), copying the
  full catalog several times per request): ~31us/164KB -> ~0.8us/0.3KB at 300
  models. Deduplicated the redundant name/type scans in resolveQualifiedSelector.

JSON decoding (goccy/go-json):
- Migrate internal/ + cmd/ from encoding/json to github.com/goccy/go-json
  (drop-in; package is named json). gjson is unchanged.
- ~3.8x faster realistic chat-body decode with fewer allocations.
- goccy is slightly more lenient than encoding/json on a couple of malformed
  inputs (leading-zero numbers; malformed values in skipped passthrough fields).
  Accepted under the gateway's accept-generously principle and pinned by
  TestDecoderLeniencyIsBounded.
- Drop the redundant gjson.ValidBytes walk in extractUnknownJSONFields (callers
  already validate via the preceding Unmarshal).

Request snapshot allocations:
- Add NewRequestSnapshotWithOwnedMaps so ingress capture owns the freshly built
  route/query/trace maps and body, cloning only the live header map.
- Add HeadersView (zero-copy) and route read-only callers to it.
- Remove the now-superseded NewRequestSnapshotWithOwnedBody constructor.

Perf harness:
- Make the gateway hot-path benchmark exercise the real Router + populated
  catalog (it previously bypassed routing, giving false confidence) and add a
  guard case for it. Add a resolution micro-benchmark.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* test(perf): tighten hot-path allocation guard thresholds

CI (linux/amd64) and local (darwin/arm64) produce identical allocation counts
and near-identical byte counts, confirming these are deterministic. Tighten the
ceilings from "intentionally generous" to ~10% over the measured baseline so the
guard catches smaller regressions while still absorbing Go/dependency drift:

  hot_path:          125 -> 120 allocs            (baseline 113)
  routed:            160 -> 150 allocs, 18->16 KB (baseline 137 / ~14.7 KB)
  responses_stream:  310 -> 222 allocs, 25->22 KB (baseline 202 / ~19.6 KB)
  shared_observers:  unchanged (already tight, no headroom to trim)

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(providers): address PR review on selector index and benchmarks

- registry: trim publicName/ProviderType when building the qualified-selector
  index and skip empty keys, matching the trimmed lookup inputs and the previous
  catalog scan (which compared trimmed fields on both sides). Prevents the O(1)
  fast path from missing when provider metadata carries whitespace padding.
- resolve_bench_test: build exactly totalModels (round-robin across providers)
  instead of providersN*(n/6); the models=50/1000 cases previously benchmarked
  48/996 models due to integer truncation. Add benchSelector helper.
- request_snapshot_test: extend the owned-maps test to assert route/query/trace
  maps are owned (not cloned) while headers are still defensively cloned.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>

feat(dashboard): support fully offline use (#408)

* feat(dashboard): serve fonts and JS libraries locally for offline use

The dashboard loaded Inter (Google Fonts) and chart.js, Alpine.js, and
Lucide (jsDelivr CDN) over the network, making it unusable offline.

Vendor all of them into the embedded static tree and reference them via
the existing assetURL pipeline:
- static/vendor/: chart.js, Alpine.js, Lucide (verified against the SRI
  hashes previously declared in layout.html; sourceMappingURL comments
  stripped so no .map files are requested).
- static/fonts/: Inter as a single variable woff2 per subset (latin,
  latin-ext) with a local @font-face stylesheet.

Extend the //go:embed directive and drop the preconnect/CDN/integrity
attributes. No build system or asset-serving changes required.

Refs #407

* fix(admin): echo pagination in disabled-reader log responses

When audit logging (LOGGING_ENABLED, off by default) or usage tracking is
disabled, the /admin/audit/log and /admin/usage/log fast paths returned an
empty response with limit:0. The dashboard reuses the returned limit as the
next request's limit param, so the following request sent limit=0, which
validation rejects with 400 "invalid limit, expected positive integer".

Echo the requested limit/offset in both disabled-reader responses so they
match the enabled-reader contract (which clamps limit to a positive value).
The dashboard's audit/usage pages now load cleanly with their readers off,
showing an empty list instead of erroring.

Refs #407

* fix(admin): default page size in disabled-reader log responses

Address PR review: when the caller omits limit, params.Limit is 0, so the
disabled-reader fast paths still echoed limit:0 and could drive the same
client limit=0 loop. Echo the documented default (25 audit / 50 usage) via
named constants alongside the existing max constants, matching the
enabled-reader clamp. Tests now omit limit and assert the default limit and
offset. Also harden the offline-resource guard to catch protocol-relative
(//cdn...) URLs.

Refs #407