Context
Federated backend auth (#147) caches the temporary credentials the proxy gets from AssumeRoleWithWebIdentity so it doesn't re-mint a JWT + re-call AWS STS on every request. Today that cache is in-memory and per-isolate:
It does not use the Cloudflare Cache API, KV, or Durable Objects.
The gap
The cache does not span Worker instances:
- Each isolate mints + exchanges on its first request per role; different isolates (other colos, parallel isolates, or a recycled isolate) each do their own first call.
- The cache is lost when an isolate is evicted.
- There is also no single-flight: concurrent cold requests for the same role each do their own STS exchange.
This is fine for busy connections (isolates are long-lived and reused, creds last up to the role session duration), but wastes STS calls / adds first-request latency for cold or low-traffic connections, and offers no headroom against STS volume/rate limits at scale.
Options for cross-instance caching (if/when needed)
| Option |
Fit |
Catch |
| Workers KV |
Natural distributed cache, keyed by role ARN |
Eventual consistency (needs a generous expiry margin); temp creds at rest in KV. |
| Durable Object |
Strong consistency; enables single-flight |
Extra hop/latency; more moving parts. |
Cache API (caches.default) |
— |
Poor fit: caches Response objects, per-colo (not global). Not recommended. |
Security caveat (all options): this persists short-lived credentials outside the isolate. They're short-TTL and KV/DO are encrypted at rest, but it widens the blast radius — a deliberate decision, not a default.
multistore constraint: CredentialCache is in-memory only and not pluggable (no cache-backend trait). A KV/DO cache means either caching at the proxy layer (wrap the exchange, bypassing multistore's cache) or adding a pluggable cache trait upstream in multistore.
Recommendation
Keep the per-isolate in-memory cache as the default — it's cheap, correct, and captures most of the benefit. Treat cross-instance caching as measure-first: once federation is live, if STS call volume / cold-start latency shows up as a real problem, add KV (tight TTL + expiry margin) or a Durable Object (if we also want single-flight). Don't build it speculatively given the security tradeoff.
Refs
Context
Federated backend auth (#147) caches the temporary credentials the proxy gets from
AssumeRoleWithWebIdentityso it doesn't re-mint a JWT + re-call AWS STS on every request. Today that cache is in-memory and per-isolate:CredentialCacheis aHashMap<role_arn, creds>behindArc<Mutex>, and we hold theOidcCredentialProviderin aOnceLockso it survives across requests within one Worker isolate (relies on feat(oidc-provider): make OidcCredentialProvider cheaply cloneable (shareable warm cache) developmentseed/multistore#69 making the providerClonewith anArc-shared cache).It does not use the Cloudflare Cache API, KV, or Durable Objects.
The gap
The cache does not span Worker instances:
This is fine for busy connections (isolates are long-lived and reused, creds last up to the role session duration), but wastes STS calls / adds first-request latency for cold or low-traffic connections, and offers no headroom against STS volume/rate limits at scale.
Options for cross-instance caching (if/when needed)
caches.default)Responseobjects, per-colo (not global). Not recommended.Security caveat (all options): this persists short-lived credentials outside the isolate. They're short-TTL and KV/DO are encrypted at rest, but it widens the blast radius — a deliberate decision, not a default.
multistore constraint:
CredentialCacheis in-memory only and not pluggable (no cache-backend trait). A KV/DO cache means either caching at the proxy layer (wrap the exchange, bypassing multistore's cache) or adding a pluggable cache trait upstream in multistore.Recommendation
Keep the per-isolate in-memory cache as the default — it's cheap, correct, and captures most of the benefit. Treat cross-instance caching as measure-first: once federation is live, if STS call volume / cold-start latency shows up as a real problem, add KV (tight TTL + expiry margin) or a Durable Object (if we also want single-flight). Don't build it speculatively given the security tradeoff.
Refs
Clone/Arc-shared cache: feat(oidc-provider): make OidcCredentialProvider cheaply cloneable (shareable warm cache) developmentseed/multistore#69