What

Configure several API keys for one provider and GoModel spreads outbound requests across them round robin. Each key carries a fraction of the traffic, raising the per-key rate limit you hit before a provider starts returning 429.

OPENAI_API_KEY=sk-first
OPENAI_API_KEY_2=sk-second
OPENAI_API_KEY_3=sk-third

Or in config.yaml:

providers:
  openai:
    type: openai
    api_keys:
      - "${OPENAI_API_KEY}"
      - "${OPENAI_API_KEY_2}"

User-visible impact

Rotation defeats provider prompt caching. Providers scope their prompt cache to the key that filled it, so consecutive requests under different keys never hit a warm cache. Rotate to lift rate limits, not to save cost. This is called out in a <Warning> at the top of the new docs page, in .env.template, and in config.example.yaml.

Nothing changes by default. Rotation is off until a second key is configured; one key behaves exactly as it always has, warm cache included.

Design

Every provider already resolved its credential through a per-request header hook — llmclient calls headerSetter inside buildRequest, which sits inside the retry loop. The credential never had to be a string captured at construction; it just always was one.

So rotation hooks in there. providers.Keyring (nil-safe, atomic round-robin) replaces the captured key string, and the factory builds one ring per provider instance into ProviderOptions.Keys. Providers pick it up with opts.Keyring(apiKey), which falls back to a single-key ring when constructed outside the factory (tests, NewWithHTTPClient). No provider constructor signature changes. Because every client a provider builds shares the one ring, rotation stays even across all of its endpoints.

This also let CompatibleProvider.apiKey — a write-only dead field — become the thing that carries the rotation.

Provider-specific behavior

• Rotates: chat/responses/embeddings/audio/images/batches and provider-native passthrough (all share the header hook), plus the semantic-cache embedder, which has its own auth path. Embeddings have no prompt cache to lose, so rotating there is pure upside.
• Realtime websockets pick a key when the session opens and hold it for the session's life.
• Retries advance the ring, so a request throttled with 429 retries under a different key rather than hammering the one just throttled.
• Unaffected: keyless providers (Ollama, vLLM) and providers that authenticate another way (Vertex, Bedrock ignore these vars; Gemini rotates only on its api_key auth branch, not Vertex OAuth).

The ambiguity worth reviewing

The trailing number collides with the existing suffixed-provider mechanism. The parser disambiguates, and both cases are pinned in tests and verified in the running binary:

_1 is accepted as a synonym for the unsuffixed key rather than silently ignored — dropping a key someone deliberately set is a costly footgun, and Postel's Law is this repo's first principle.

Unresolved ${VAR} placeholders are dropped rather than forwarded as literal credentials; a provider left with no keys is not registered, exactly as before.

No DB schema

Keys come from env/YAML like every other provider credential. The only state is an in-memory counter, so there's no table and no migration — persisting secrets for zero functional gain. Counters are per-process and reset on restart, which is fine: the goal is to spread load, not to sequence it exactly.

Verification

Beyond unit tests, I ran the real gateway against a mock upstream that logs the Authorization header it receives:

• Three keys → upstream saw sk-alpha → sk-beta → sk-gamma cycling exactly.
• One key → same key every time (prompt caching preserved).
• OPENAI_REGION_2_API_KEY → registered a separate provider, not a rotation slot.

Tests added: keyring_test.go (incl. a concurrency test asserting an even split under -race), keyring_config_test.go (env/YAML resolution, numeric-not-lexical slot ordering, dedup, gaps, the suffix ambiguity above), and openai/keyrotation_test.go (end-to-end header assertions incl. retry-uses-next-key and keyless-sends-no-credential).

Full suite, -race, and golangci-lint clean.

Docs

New page docs/providers/key-rotation.mdx registered in the Providers tab, cross-linked from providers/overview. Includes a comparison table steering users who want several keys and prompt caching toward suffixed providers behind a round_robin virtual model.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Added provider API key rotation using multiple keys per provider with round-robin selection via numbered environment variables and api_keys in YAML.
  • Added documentation and examples for setup, numbering rules, and behavior (including that rotation disables provider prompt caching).
• Bug Fixes
  • Improved request and realtime/websocket authentication so retries and live sessions use the correct rotated key.
  • Updated configuration handling to drop empty/unresolved keys and avoid registering providers without usable credentials.
• Documentation
  • Expanded guides, provider overview notes, and template/config examples to reflect multi-key configuration.