Summary

This PR refactors OmniRoute's entire circuit-breaker and cooldown stack for release/v3.7.0.

Before this work, resilience behavior was spread across too many overlapping layers: connection cooldowns, provider breakers, model-availability state, combo-specific health state, and dashboard/API-specific views of runtime health. The result was hard to reason about and hard to trust: the same failure could be interpreted differently depending on where it was handled, and different surfaces could drift away from the actual runtime behavior.

The goal of this refactor is to give OmniRoute one clear resilience model with clear responsibilities at each layer, so the runtime, dashboard, MCP, API, and docs all describe the same system.

What the new system is for

The rebuilt resilience system is split into four layers. Each layer exists to answer a different operational question.

1. requestQueue

What this layer is for

This layer protects a single connection/account from being hit by overlapping local requests when that connection should process work sequentially.

What it means operationally

• if a connection is healthy but already busy, requests can be serialized instead of racing each other
• this is about local concurrency control, not failure handling
• it prevents self-inflicted contention on the same account

2. connectionCooldown

What this layer is for

This layer decides when one specific connection/account should temporarily step out of rotation after a retryable failure.

What it means operationally

• if one account is rate-limited (429) or temporarily unstable, OmniRoute cools down that connection instead of repeatedly hammering it
• this is connection-scoped protection, not provider-wide protection
• the runtime uses baseCooldownMs, upstream retry hints, and bounded backoff to decide how long that connection should rest
• connection-scoped 429 rate limits stay here and do not trip the provider breaker

3. providerBreaker

What this layer is for

This layer decides when a provider as a whole is unhealthy enough that OmniRoute should stop routing traffic into it for a while.

What it means operationally

• this is the provider-wide safety gate
• if failures are no longer isolated to one bad account and look like provider-level instability, the breaker opens
• the breaker tracks provider-wide final transient failures after fallback exhaustion (408, 500, 502, 503, 504)
• connection-scoped 429 rate limits do not contribute to the provider-wide breaker
• direct requests and combo routing use the same provider-level breaker state
• the Health page is the runtime view of this provider-wide state

4. waitForCooldown

What this layer is for

This layer decides what OmniRoute should do when every eligible connection is temporarily unavailable only because of cooldowns.

What it means operationally

• OmniRoute can either fail fast or wait briefly for the earliest cooldown to expire
• this is the policy layer for cooldown-aware retry behavior
• it prevents the system from either giving up too early or retrying forever

What combo routing is responsible for

Combo routing is now treated as a routing layer, not as a second resilience system.

That distinction is important.

A combo should answer questions like:

• which providers/models are candidates for this request
• in what order or strategy they should be tried
• how intelligent routing should score or prioritize those candidates

A combo should not maintain its own parallel understanding of provider health, cooldown state, or breaker state.

What this means operationally

• combos use the same provider breaker state as direct model requests
• combos use the same connection cooldown/account availability state as the rest of the runtime
• combo fallback happens on top of the unified runtime resilience state, not beside it
• intelligent combo panels focus on routing inputs such as candidate pool, mode pack, and exploration rate
• runtime health stays on the Health page instead of being reinterpreted inside combo-specific UI panels

In other words: combo decides where to try next; the resilience system decides whether that target is currently eligible.

Runtime behavior after the refactor

The runtime now follows one consistent flow:

1. queue locally when the selected connection should not run concurrent work
2. cool down individual connections when a failure is connection-scoped and retryable
3. open the provider breaker only when failures are provider-scoped after fallback exhaustion
4. let combo routing choose among the remaining eligible targets using its configured strategy
5. decide whether to wait or fail when all remaining candidates are only cooling down

That gives each decision a single home instead of spreading the same responsibility across multiple overlapping subsystems.

Account and fallback handling

This refactor also makes the fallback path easier to reason about.

• cooldown writes, account suppression, and model lockouts all flow through the shared auth/fallback path
• connection-scoped 429 rate limits stay in cooldown/backoff and wait-for-cooldown behavior instead of being counted as provider-breaker failures
• provider-specific project-routing failures are treated as retryable provider responses instead of terminal account death
• OAuth invalid-token responses stay in the provider recovery path instead of being flattened into a generic terminal state too early

Dashboard, API, and MCP behavior

The non-runtime surfaces now describe the same resilience model instead of inventing their own parallel layers.

Dashboard

• the Resilience settings page exposes the same four-part model used by the runtime
• the Resilience settings copy explicitly describes connection-scoped 429 handling as part of Connection Cooldown
• the Health page shows provider-wide breaker state
• intelligent combo panels show routing configuration inputs, not a second runtime health model
• provider pages reflect provider/connection state from the unified runtime model

API

• /api/resilience exposes the same settings shape used by the runtime and dashboard
• connectionCooldown is configured with:
  • baseCooldownMs
  • useUpstreamRetryHints
  • maxBackoffSteps
• providerBreaker only counts provider-wide final transient failures; connection-scoped 429 handling stays outside this layer

MCP

• MCP dashboard presets use the same resilience structure
• advanced MCP resilience tools switch profiles using the same model

Documentation

The docs now describe the same 3.7.0 resilience model used by the implementation:

• README.md
• docs/API_REFERENCE.md
• docs/ARCHITECTURE.md
• docs/USER_GUIDE.md
• docs/openapi.yaml

Validation

Type and test coverage

• npm run typecheck:core
• node --import tsx/esm --test tests/integration/integration-wiring.test.ts
• node --import tsx/esm --test tests/unit/autocombo-unification.test.ts
• node --import tsx/esm --test tests/unit/account-fallback-service.test.ts
• node --import tsx/esm --test tests/integration/chat-pipeline.test.ts
• git push pre-push hook: npm run test:unit (3240 passing tests)

Playwright coverage

Playwright coverage was run in dev-server mode and verifies the behavior of the new model across UI surfaces:

• resilience settings UI with the 3.7.0 connection cooldown fields
• Health page provider breaker rendering for multiple provider states
• provider dashboard behavior under the unified provider/runtime model
• intelligent combo panels showing configuration-oriented routing inputs

Commands used:

• OMNIROUTE_PLAYWRIGHT_SERVER_MODE=dev npx playwright test tests/e2e/resilience-plan-alignment.spec.ts
• OMNIROUTE_PLAYWRIGHT_SERVER_MODE=dev npx playwright test tests/e2e/combo-unification.spec.ts

Note

Playwright was executed in dev-server mode because the current branch still has unrelated Next build/prerender failures in start/build mode (/_global-error, /callback). The behavior covered above was validated successfully in dev mode.